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ABSTRACT 



This guide describes the New York State assessment 



process as a framework foi other districts aiming to develop an 
effective approach to performance assessment. In 1984-85 the New York 
State Board of Regents initiated the Comprehensive Assessment Report 
(CAR) as a process of reporting and using education results 
throughout the state. The conceptual approach of the guide conceives 
of standardized test data as a limited first step in assessing school 
effectiveness. In going beyond test data, one must look for 
convergence of test data and other indicators of school performance 
and one must separate the data into component parts to gain 
information about subgroups of students and schools. Although 
technical statistical considerations are not emphasized in this 
guide, it is essential that comparisons between schools, districts, 
or groups take into account demographic and other salient 
characteristics of the groups being examined. The introduction 
establishes the framework for this guide. The second chapter 
describes the CAR, its purposes and processes, and the data it 
includes. Chapter 3 discusses the limitations of the data and 
possible misuses. Chapter 4 generalizes from the New York experience 
to indicate how a district can prepare and present a comprehensive 
assessment report. Appendix 1 contains a model assessment report, and 
Appendix 2 focuses on basic measurement concepts. Appendix 3 presents 
a directory of school improvement resources, and Appendix 4 includes 
a 107-item bibliography of research on school effectiveness. Five 
data tables and 12 graphs are included. (SLD) 
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Chapter 1 



INTRODUCTION 



As part of jthe nationwide emphasis on school improvement, 
schools and districts throughout the country are being asked to 
assess student achievement, to present data for public review, 
and to initiate district planning procedures to enhance school 
effectiveness • 

In many instances, this attention to formal assessment has 
been a direct response to mandates emanating from state education 
agencies or state legislatures- Confronted with widespread 
concern about a, decline in education standards and growing 
demands for accountability, numerous states throughout the 
country have initiated reforms that require and support public 
disclosure of student achievement data. 

Presenting school/student achievement data is obviously not 
without risk even in the most politically stable school 
districts. The possibility that users will misinterpret the data 
are considerable, as are the possibilities that the data will 
correctly suggest problems that are politically costly to 
acknowledge and resolve. 

Advocates say, however, that states and districts that open 
themselves to such scrutiny are taking a critical step toward 
unifying and consolidating community support for their schools 
and their school improvement efforts and strengthening the 
partnership essential for progress. 

Expressing that hope, in 1984-85 the New York State Board of 
Regents initiated the Comprehensive Assessment Report (CAR) . The 
CAR, which is part of the Regents Action Plan to Improve 
Elementary and Secondary Education in New York State, provides a 
process for reporting and using education results throughout the 
state. 



Parpose of the Guide 



This guide will describe in considerable detail the scope 
and limitations of the New York State assessment process and 
steps that can be taken to amplify CAR data and make them more 
useful for school improvement purposes. Our intention is both to 
create a practical guide of immediate value to schools and 
districts in New York State and to use the New York State 
experience to frame and highlight issues that could usefully be 
considered by districts outside the state aiming to develop an 
effective approach to performance assessment. 

We have prepared the- text with the needs of district-level 
administrators in mind. However^ building administrators^ 
teacherSf school improvement team members^ and parents may also 
benefit from the discussion. The presentation is largely 
nontechnical and the approach stresses questions that one should 
ask when examining performance data. 

Conceptual Frameworic 

Most guidebooks begin by providing a map of the territory 
the authors consider key to the phenomena they are discussing. 
Before launching into the sxibstance ^of this guide, we would 
similarly like to provide a general framework to hexp readers 
understand why we suggest the things we do. 

We see analysis of standardized test data as a limited first 
step in assesr^ing school effectiveness - as an important but not 
sufficient indicator for determining whether schools are working 
effectively with students. To make effective use of 
standardized test data one must go beyond them. One must look 
for convergence between test data and other indicators of school 
performance, such as teachers* subjecrive reports of how students 
are doing. One must also carefully "disaggregate the data'* or 
separate them into their component parts so that they provide 
information about sxibgroups of students and schools. When data 
are looked at only in summairy form one often misses important 
differences that may exist by race, gender, students* poverty 
status, etc. 

You will note when reading this guide that we very 
cautiously mention statistical procedures that districts can 
perform to determine whether obtained differences in student 
performance are significant mathematically, that is, unlikely to 
have occurred by chance. We don*t stress these technical 
statistical considerations, however, for two reasons: fi:rst 
because a preoccupation 'with determining statistical significance 
may inhibit districts from a common sense examination of the 
data; second because when districts find similar trends suggested 
by several standardized tests and other indicators across time, 



this information will probably provide a meaningful basis for 
action. 

What we do emphasize is that when schools, districts, and 
student groups are compared, those comparisons must take account 
of the demographic and other salient characteristics of the 
groups being examined. Without this check, it would be very 
difficult to identify schools with a high percentage of 
academically able students that are failing to maximize those 
children's potential. Similarly it would be difficult to 
identify schools with a high proportion of academically 
disadvantaged students that are making great progress with their 
youngsters . 



Organization of the Guide 



The guide is divided into four chapters • The first is this 
Introduction, which establishes the framework for the report* 
The second chapter describes the New York State Comprehensive 
Assessment Report (CAR), its purpose and processes, and the data 
included in the CAR. Chapter Three discusses the limitations of 
the data and possible misuses. Chapter Four generalizes from the 
New York State experience and details how a district might 
prepare and present a comprehensive assessment report. Such 
issues as data verification, supplementation, interpretation, and 
illustration are addressed in that chapter. 

Appended to the guide are sections that present useful 
information and resources to support the comprehensive assessment 
and school improvement process. Appendix 1 contains a model 
assessment report. Appendix 2 focuses on understanding normative 
test data. Appendix 3 presents a directory of school improvement 
resources, including books, manuals, and organizational 
resources. And Appendix 4 is a bibliography of school 
effectiveness research. 
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Chapter 2 

THE NEW YORK STATE COMPREHENSIVE ASSESSMENT REPORT (CAR) 



Piirpoae of the CAR 

The Comprehensive Assessment Report (CAR) is an essential 
part of the Regents Action Plan to Improve Elementary and 
Secondary Education in New York State* 

The New York State Board of Regents expressed the hope that 
this opportunity for public review and discussion of student 
achievement would serve two ends: 

• increase community involvement , in education and 

• assist district staff in planning for school improvement* 

As stated in the Regents Action Plan; 

Riblic review and debate about student adiievetoent has 
proven to be one of the most effective ways to develop the 
oonoem of parents and the public and to bring about action* 
i^xxa exp&i&nce gained frqn the O^^artxnent^s Rescuroe 
Allocation Plan (PAP) , one of the most effective ways for 
drawing attention and involvement of oonzunities and boards 
of education is to display trend information on pupil 
^progress* The informatlcai is itself a pcwerful 
motivation to take action. 



The CAR Process 

Each year the New York State Education Department (BED) 
administers a number of standardized testing programs and olDtains 
demographic, and other data from each of its schools. These data 
are entered into the Basic Education Data System (BEDS). The CAR 
integrates much of this information. The data are aggregated at 
the school and district levels for three successive school years. 
Achievement, demographic, and other data are presented in a 
report to local districts. The districts are then required to 
review the information and report it to their Boards of Education 
and members of the education community. 

In addition to serving as a mechanism for initiating school 
improvement planning at the local district level, the CAR data 
are used by the state to identify schools **most in need of 
assistance." Schools scoring in the bottom 10 percent of the 
distribution on one or another of the CAR criteria are identified 
for special attention. Such schools are required to develop a 
comprehensive school improvement plan (CSIP) . The format of the 
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plan is prescribed by the Cosunisffloner. The plans themselves are 
developed by the schools with the assistance of State Education 
Department ataff assigned to the particular school. Following 
completion; each plan must be approved by the local Board of 
Education and submitted to 'the State Education Department no 
later than April 30, with Implementation to begin the following 
September • 



Data Included in the CAR 



The Comprehensive Assessment Report developed by the State 
Education Department provides three-year trend data that Includes 
the following information: 

• measures of student achievement; 

• demographic Informaf'on; c^nd 
m school characterise cs. 

The components of each data set will be elaborated fully in this 
chapter. 



Measures of Student Achievement 

The CAR presents two forms of student achievement data: 
standardized tests and high school graduation results • 

» 

Standardized Tests ^ The New York State Testing Program is 
designed to assess student performance on a regular basis 
throughout the educational process. One of the primary purposes 
of this testing program is to assist schools in identilyin? 
students with learning difficulties. 

Th3 Comprehensive Assessment Report Includes results from 
the Pupil Evaluation Program (PEP) , the Program Evaluation Test 
(PET) , the Preliminary Competency Tests (PCTs) , the Regents 
Competency Tests (RCTs) , and the Regents Examinations. 

The PEP tests are administered at the elementary school 
level, the PCTs at the middle school level, and the RCTs and 
Regents Exams, at the secondary level. Tests at the elementary 
and middle school level focus on reading, writing, and 
mathematics skills. The high school level exams measure academic 
achievement as well as basic skills competen /. 



Pupil Evaluation Program fPEP> . Students in grades three and 
six take PEP tests in reading and math. In grade five, writing 
skills are assessed. The reading and math tests contain 
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multiple-choice items th^t are revised every three years. The 
writing test is a two-paii: essay exam that is graded by local 
teachers in accordance with the state's holistic scoring 
guidelines* 

The table that follows presents sample district and building 
sxammaries for the New York State Pup:*l Evaluation Program. 



COIMCHCMIIVC ASSCSSMCtcr RCmT 
OCTMCR tfl7 

NCW YOW STATC Wll CVAUMTION HIOQfUN: OISTRia M tMILOIHO SOmMIKS 

SCHOOL OlSTRtCT COOC: 

^iJ ^!5!i°** INTO^ATIOM AMUT TXC ^CftfONMANCC Of Tm OWTHICT'S W|U$ ON TXI WflU CV«.U*TIOM PROGMAM T '.TS fon TH€ 

iJi^.^'i SUWMIir OATA AAC MOVIOCO fOU Tm T0TA4. OISTHICT *m fOU CACH tUlLOlM THAT HAS OAAOCS 3 S. ANO.Oil • 

!2L'J2L**^5^ ^ T^^*-' '♦•^ ^ W«OtlJ«WT or TM« OTAOC in MNICN CACH TCST was MHINISTCRKO. THC NUMiCR Of fUPtUS WK) 

igy. j^.l!?y-- ^HOSC TCSTCO «M0 SOOHKO AWVC Tm STATC MfCROfU rOIKT (SMI rot THC TCST. OATA AAC PAOVloeO 
foil NOMIANDICA^KO PUPILS ONLY. 

I!SiIfi?!J*,2! 'Vlii* ^m^^ ^ ^^Sa? APPtOXinATCUY KQU/u. TO TMf OAAOC KNAOLLMCNT THC -fCACtHT TCSTM AiOVC SAf 

iS^SliJj?* ^^^^ ^ '•^^ WOOACiS IN OCVKUOAIHO TH5 IaSIC SKXUUS Of RKAOiNO COl#RKHCNSION. HATHCIUTtCS. 

AMD MITtMB. PUPIU SGONINt tCLOf THC SAP It^ST tC fHOVIOCO WITH ftCMCOIM. INSTAUCTION. 



C GRAOC 


NAUC Of TCST 


CNAOtU. 


NUHtCA 

TCSTCO 


PCACCNT 
TCSTCO 
AMVC SAP 


CMIOU. 


tvmin 

TCSTCO 


PCRCCNT 
TCSTCO 
AAOVC SRP 


CNMLL. 


ttl^-|7 

tesTCo 


TeSTCO 
AIOVC SAP 


TOT At OlSTAICT 
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3 


RCAOlNa 


CI1 . 


•71 




•41 


•43 


•Ok 


849 


640 


8tX 




MATH 


•It 


•74 


iS 


•4S 


• !• 


•1 


64S 


140 


ft 


> S 


WRiTINa 


711 


701 


10 


•70 


••• 


•s 


43t 


62i 


94 
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ACAOl>« 


720 


713 


•0 


710 


707 


!• 


615 


614 


II 




MATH 


730 


711 


•4 


710 


701 


•3 


•IS 


613 


98 



SCHOOL 
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RCAOlNa 


71 


71 
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73 


73 


ir. 


60 


91 


9iX 




MATH 


71 


88 
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72 


88 


too 


90 


60 


too 
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WRiTlNa 


78 


78 


18 


48 


48 


12 


69 


69 


too 
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RCAOlNa 


7S 


74 


84 


71 


'ii 


77 


60 


60 


93 




HATH 


7S 


79 


•3 


71 


71 


98 


60 


60 


91 
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91 
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SO 


SO 
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60 
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81 
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90 


SO 


94 
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97 


too 
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71 


71 


•3 


70 


70 
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94 


S4 


«• 
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RCAOINO 


70 


70 


to 


88 


8S 


91 


81 


61 


92 




MATH 


70 


• 1 


94 


88 


88 


91 


•1 


61 


99 



As the table indicates, for each grade and test the CAR presents 
the following information from the PEP tests: 

• EnroIInent: The number of non-handicapped students in 
the specified grade at the time of the test. 



• Number Tested: The number of students who actually took 
the test. Absences as well as exemptions due to limited 
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English proficiency account for the difference between 
"enrollment" and "number tested." 

m Pazxrent Tested Above SRP: The percent of students 
making normal progress in developing the basic skills. 
The State Reference Point (SRP) is the staters criterion 
of satisfactory performance. The SRP is based on the 
SED*s calculation of test score averages across the state 
during the first year the exeua is administered. A score 
below th^ SRP is an indication that a student requires 
special help or remediation in the skill area. Students 
scoring below the SRP must be provided with remediation. 

Proc^am Bvaluat ion Tests fPETs) . The PET in social studies 
is administered annually to pupils in grade 6 beginning in the 
1986-1987 school year. The. test is designed to measure the 
effectiveness^ of the elementary school stocial studies programs of 
public and nonpublic schools within the state. The test yields 
data that can be used in the planning, management, and evaluation 
of programs at both state and local levels. The test contains 
three components, two of which are required and are reported on 
the CAR. The two required components are the objective test, 
consisting of fifty questions, and the writing sample, involving 
one essay. The third component, the participation project, is 
optional. The participation project offers schools an 
opportunity to assess how well pupils aj^ply content, concept, and 
skill learning in a group problem solving situation. 

The CAR contains data for the PET for the total district as 
well as for each school that administered the test. Data are 
presented for nonhandicapped pupils only. Three columns of data 
are provided. The first coliimn indicates the number of 
nonhandicapped pupils tested. The second column indicates the 
mean score for the pupils tested. The third column indicates the 
percentile rank associated with the mean score. The percentile 
ranks are based on a frequency distribution of mean scores for 
all public and nonpublic schools in New York State administering 
the grade 6 social studies test. Thus, when percentile ranks are 
reported for a public school district, the data represent the 
percentile ranks of the "average school" in that district. 

PT;'e'^^T^i narY CoMpetency Tests fPCTsK These tests, given to 
students in either grade 8 or 9, are also designed to assess 
basic skills in reading, mathematics, and writing. Like the PEP 
tests, their primary purpose is the identification of pupils who 
need special help if they are to develop a basic level of 
competenca in these critical skill areas-. 

PCT results are presented in the CAR in the manner described 
for the PEP, however the PCT results include tbe Percent of 
Enrollment Above SHP ^ rather than the Percent .sted Above SRP > 
This statistic shows the percentage of students in the entire 



grade who scored above the state reference point. This percent 
includes those pupils who scored above the SRP on the Preliminary 
Competency Tctst as well as those who were exempted from the 
testing program because their competency had been established on 
other tests. 

aeaenta CoMPetencv Tests fRCTgK In order to receive a . 
local high school diploma, pupils must demonstrate competency in 
reading, writing and mathematics. The Regents Competency Tests 
(RCTs) provide one means for demonstrating the required level of 
proficilency. The first Regents Competency Test in mathematics 
was administered in June of 1979 and the first Regents Competency 
Tests in reading and writing were administered in January 1980. 
These tests established new standards of achievement in basic 
skills for pupils receiving high school diplomas beginning with 
the class of 1981. The Regents Action Plan specifies mandates 
for competency tests in new^ curriculum areas. As of 1991, 
students will also be required to demonstrate competence in 
occupational education, science, global studies, and American 
government . 

In most z^^^"1b throughout New York State', the RCTs are 
taken only by those pupils who do not take Regents Examinations, 
students who are enrolled in Regents level or advanced courses 
will normally choose to demonstrate competency by successful 
completion of the Regents examinations in English and Math. 

Like the PEP and PCT programs, the RCTs consist of three 
tests in the same basic skills areas: reading, writing, and 
mathiematics . The RCTs in mathematics are normally taken in the 
ninth grade but may be taken any time from grades nine to twelve. 
ThG RCTs in reading an writing may be taken only by pupils in 
grades eleven or twelvr . Students who fail the RCT may repeat 
the test at a later da^e. In the interim, the district is 
rec[uired to provide appropriate remedial instruction. 

Like the tests in the PEP and PCT groups, the RCT reading 
and math tests have a multiple-choice format. The writing test 
requires students to complete three writing tasks, each of which 
is assessed by one of three raters. Those writing tests that 
receive passing grades are then forwarded to SEO for a second 
review and final pass/fail decision. 

As with the PEP, PET and PCT reports, the reports for the 
Regents Competency Tests also provide information for the entire 
district and for individual buildings for each of three years. 
The results reported are the total number of students tested 
( Number Tested ) and the percent of students who passed ( Percent 
Passed), that is, scored above the state reference point. The 
exams are given three times a year, in January, June, and August, 
but only January and June scores are reported. 
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Regents ^ ftp^ yiai^iona ^ Regents Examinations are achievement 
tests based oh the courses of study recommended by the State 
Education Department for grades 9-12* Tests are currently 
offered in" English ^ mathematics^ foreign languages^ sciences^ 
social studies, and business. 

Many high schools have rigorous admissions requirements for 
Regents courses , and the exams are taken primarily by 
college-bound students. The present Regents Examination Program 
began in 1878 , and the tests have traditionally been used to 
gauge teaching effectiveness and learning success and to 
determine eligibility for the special Regents honors diploma* 

Students may repeat any of the exams if they fail» Each 
exam, is developed by the SED, working closely with classroom 
teachers. Most, Regents Examinations combine multiple choice with 
essay questionsr but the tests in chemistry , earth science, and 
physics are entirely multiple choice. All Regents Examinations 
are: scored by local classroom teachers; random samples from each 
discipline are sent to the SED for a second review. 

The CAR presents Regents Exeunination results for each 
individual test. As with the other statewide tests, information 
is presented for the total district and for individual school 
buildings. The report also indicates the number of students 
tested f Number Tested ) and the percent passing of the total 
number teisted f Percent Passing of No. Tested) . In addition, the 
Regents Examination report also includes an additional column - 
Percent of Enrollment Tested . This number represents the 
relationship between the ntamber of pupils taking the exam and the 
average enrollment for that particular grade. For example, 
assume that 200 tenth gradie students completed the Social Studies 
test in June. In June, the total enrollment in that class was 
400. Therefore, the Percent of Enrollment Tested would be 50%, 
i.e. 200 students out of a total of 400 student were tested at 
that time. 

The following table presents an overview of the tests 
included in the CAR: 



Overviev of Tests Included in the CAR 



Test Skill Area Grade Level 



Pupil Evaluation Program 


Basic Skills Testing: 
Reading 

Writing 


3,5, and 6 
3 and 6 
J ana o 
5 


Program Evaluation Test 


Social Studies 


6 


Freiimmary competency Tes^ 


r^^sic sxiiis Tescing 
Reading 
Writing 


8 or 9 
8 or 9 
8 or 9 


Regents Competency Test 


Basic Skills Testing 
Reading 
Writing 
Mathematics 


9-12 

.11 or 12 
11 or 12 
9-12 


Regents Exeuninations 


All Academic Subjects 
and Business 


9-12 



High School Graduation Results > This section of the CAR indicates 
how many handicapped and non-handicapped students in the twelfth 
grade were candidates for graduation, and how many of these 
students actually completed the requirements entitling them to 
receive a high .school diploma or certificate. The table that 
follows illustrates district and building summaries of high 
school graduation results for a saunple district. 

For each district and each high school within that district, 
the CAR lists the following for both handicapped and 
non-handicapped pupils for each of the three years: number of 
diploma candidates; number of candidates who received a local 
high school diploma; number of candidates who received a Regents 
diploma in addition to a local high school diploma; number of 
candidates who did not receive a local diploma; and number of 
pupils with handicapping conditions who received an lEP diploma 
or certificate. 

DiploBa candidates > Diploma candidates are pupils who, one 
month before graduation, should theoretically be able to complete 
all required testing and courses. For 1984-85, this figure 
includes only pupils who were diploma candidates on June 1. The 
1985-86 figure includes those who were candidates any time during 
the year, including the summer. 
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COM^RCMNSXVC ASSCSSHCKT KCPOIIT 
OCTOttA tM7 

MOM SOOOL QMOUATXON MSUCTS: OXSTHICT AND lUtLOtNQ SUMMItC£ 

SOOOL OlSTmCT COOC: 

TMtS TAtU MOVXOCS XWTOIilATXOW J<OUT 7M( OXSTRXCT'S Wilt IMO Ml CANOXOATCS fOtk GIUaMTXON OMtNQ CACH OF THC ^AST TMCC 
SO«0L YtAM. rot TW 1M4*tS SCHOOL YCAJI. SUMIAAy OATA A«£ MOVXOCO ONLY POft THC TXNC MAXOO tCTVCeH SCmMU t AMD JUMC 30 
OATA A«€ HOT mUM9 fOK PUPXU W« MCT THC ACQUXUMOfTS rOK OMOUATXOH OUAXHQ JULY AND AUOUST. rOA TMC ttl9-li AM ttli-17 
SCXOOL . YCAA$«. l UHi ff T OATA AAC MOVXOCO rOA THC TXMC KAXOO UTVCCN SC^TCMiCA 1 AM AUQUST 31. THC OATA FOA THC ttlS-li AMO 
ttM'tt SCHOOL YCAAS^^t^KUJOl ALL PUP XLS «MD ACT THC ACOUXACMMTS FQA CAAfiUATXON MXOA TO AUQUST 3t. CVCN THMGH SQMC FU^XLS MAY 
NOT HAVt AtCAXVn TmM OimjOHtt 

S m iAAY?OATA:AAC MQV10C9>0A THC TOTAL OXSTAXCT AM FOA CACH AUXLOXHO THAT CONTAXHCO QAAOC 13 PUfXLS. THC TaiC SM}tfS THC 
NUMSn or AUAXU WO MAC OXALflMA CAMXOATU AS Of JUNC 1 FOA THC 1tt4*IB SCHOOL YCAA. AM AT ANY TXMC tCTVClN SUTCMCCA 1 AM 
AUQUST 31' POA IMt-SO AM 1M0*t7. THC TAAU ALSO SHOWS THC NUHBCA Of fUfXLS mO ACCCXVCO LOCAL HXGH SCHOOL OXflOMAS OUAXNQ CACH 
SCHOOL YCAA. THC HUMACA MM ACCCXVCO ACQOfTS OXALOHAS. AM THC MA9CA MKO FAXLCO TO QAAOMTC AY THC EM Of THC SCHOOL YCAA 
Uf AAATC OATA FOA HAMXCAAACO AM N0M4M0XCAAfC0 fUAXLS AAC NOT AVAXLAALC FOA THC ttl4-IS *CMOL YCAA. 

fXNALLT. THC TAALC SHOMS THC MHBCA Of HXW SCMOL XMXVXOUALXZCO COUCATXON AAOQAAM iXCA) OXfLOM^f AM CCATXFXCATCS AWAAOCO TO 
AUAXLS MXTH HAMXCAAAXM COMXTXQM MM MCAC UNAALC TO OOMALCTC THC ' nCOUXACMCNTS FOA A ACOULAA LOCAL hXGH SCHOOL OXFLGMA ONLY 
MAMXCAAACO fUAXLS MM CAMOT CQMALCtC THC ACOUXACMCNTS FOA A ACOULAA OXFLOHA AAC CLXQXtLC FOA A HXGH SCHOOL XCF OXFLONA OA A 
CSATXFXCATC. DATA FOA THC WMCA Of AUAXLS ACCCXVXM XCA OXALQMAS AAC MT AVAXLAALC FOA THC tM4-IS SO«0!. YCAA. 

XN THC TYAXCAL HXOH SCHOOL. AAOUT 92 FCACCNT Of THC CAMXOATCS ACCCXVC A HXON SCHOOL OXFLOMA AM ASOUT 43 FCACCNT OF THC 
OAAOUATCS ACCIXVC A ACOCNTS OXFLOMA. THC FAXMAAY ACASON FOA MXSSXM GAAiXMTXON XS FAXLUAC TO SATXSFY A LOCAL COUASC ACQUXACMCNT 

••••••• 1M4-IS •••••••• ••••••• itlS-IA •••••••• ttaa-iT 

NOIAtAMX* HAMX- HOfAIAMX* HAMX* MMiAMX- HANOX- 

CA^CO CAAFCO TOTAL CA^CO CAFFCO TOTAL CAFFCO CAPPCO TOTAL 



TOTAL OXSTAXCT 



t. M. OF OXFLOMA CAMXOATCS 

pUAXM SCHOOL YCAA NA NA StS tlT 3A SS3 S«t tS 970 

3. M. OF CAMXOATCS MM ACCCXVCO 

LOCAL HXGH SCHOOL qiALOHAS NA NA tt3 117 34 t3t 901 t4 922 

3. M. Of, CAMXOATCS MM 

ACCCXytO ACQCNTS OXFLOHAS NA NA 39t 34 1 34 1 4t4 4I4 

4. M. OF CAMXOATCS MM. AY 
THC CM. OF THC SOOOL YCAA. 
FAXLCO TO SATXSFY- 

• LOCAL COUASC ACOUXAC- 

MCNTS ONLY NA NA 7A 2 2 4t t SO 

• COMFCTCNCT TCSTXM 

ACOUXACHOfTS ONLY NANAS 224 3 3 

r iOTH OF THC AAQVC 

■CQUXMCMCNrS NA NA 2 2A 28 1 i 

5. M. OF HAMXCAFFCO FUAXLS 

«M0 ACCCXVCO XCA OXFLOHAS NA 4 6 

A. M. OF HAMXCAFFCO FUAXLS 

MM ACCCXVCO CCATXFXCATCS A 



1M4-IS •••••••• «.•...• 1SI5-I8 t«««««« t9IS-l7 

DX- HAMX- N0»*4AMX- NAM)!- NOfMAMOX- MAMOX • 

CAFFCO CAFFCO TOTAL CAFFCO CAFFCO TOTAL CAF»£0 CAiiFCO r0r4L 



S H S 

1. M. OF OXFLOMA CAMXOATCS 

OUAXM SOCOL YCAA NA NA 34A 4S7 tt S?3 439 

2. M. OF CAMXOATCS MM ACCCXVCO 

LOCAL HXOH SCHOOL OXALOMAS NA NA SOS 497 tl StS 4tO 

3. M. OF CAMXOATCS MM 

ACCCXVCO. ACOCMTS OXALQMAS NA NA 23« ITS t7S tl2 

4. M. OF CAMXOATCS MM. AY 
THC CM Of THC SCHOOL YCAA, 
FAXLCO TO SATXSFY: 

- LOCAL COUASC ACQUXAf - 

MCNTS ONLY NA NA 34 3t 

- COMFCTCNCT TCSTXNO 

ACOUXACMCNTS ONLY NA NA S 

- IOTH OF THC AAQVC 

ACOUXACMCNTS NA NA t 

5. M. OF HAMXCAFFCO FUFXLS 

M ACCCXVCO XCA OXFLQMAS NA 3 

A. M. OF HAMXCAFFCO FUFXLS 

MM ACaxVCO CCATXFXCATCS 2 
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fff candidates who received a local high school 
diplo»a « The local diploma is- the basic credential signifying 
completion of a high school program. The Regents diploma, or 
Regents endorsement of the local diploma, is an honor accorded to 
thosift pupils who meet additional examination and course 
reguir^ents. The nxmber In this section of the CAR represents 
ail students who completed requirements for graduation, including 
those who received a' Regents (honors) diploma, 

iftimH<>T;> of candidates irtio received a Regents diploma in 
addition to a local high' school diploma ^ Normally, the 
requirements for a Regents diploma are more stringent than those 
for the local high school diploma. Where a student has completed 
the. additional requirements necessary for the Regents diploma, 
this diploma will be awarded in addition to the local high school 
diploma. 

jftTHnt^j^i;^ candidates who did not receive a local diploma . 
This identifies the number of students who were unable to 
complete requirements for the local diploma because they failed 
to satisfy: course requirements, competency test requirements, 
or 'both course requirements and competency test requirements, 

NtmbA-r Q-p pupils with hamdicapping conditions who received 
an lEP diploma or certif icate > Schools are required to develop 
an individualized education prog^eun (IE?) for each student who is 
classified as handicapped. This IE? establishes diploiaa 
requirements that are tailored to the specific needs and 
abilities of the student. In many cases, the handicap precludes 
completion of all of the requirements for either the Regents or 
local diploma. For these students, other requirements are 
established and the successful completion entitles them to 
receive a diploma or certificate. 



Demographic Information 

The CAR presents information about the following demographic 
characteristics of the student population, 

Racial/Ethnic Distribution and Limited English Proficiency , This 
section of the CAR report provides information about the 
re.cial/ ethnic distribution and limited English proficiency (LEP) 
status of pupils in the district and in each school, 

• Racial/ethnic distribution data include the number and 
percent of pupils belonging to each of four racial/ ethnic groups: 
American Indian, Alaskan, Asian or Pacific Islanders; Black (non- 
Hispanic) ; Hispanic; and White (non-Hispanic), 
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• LiMited English Proficiency (LEP) refers to the number 
and percent of students in the district or school who are not 
fluent in. the English language. According to the Mew York State 
COBoaissioher of Education, students with limited English 
proficiency are students who speak a language other than English 
and score below the 23rd percentile on an English language 
assessment instriiment approved by the Commissioner. The CAR 
provides xacial/ethnic distribution and limited English 
prof iciency data for the past three years. These data are 
illustrated in the table that follows. 

COMrMMCIfSIVC ASSCSSHCHT mfORT 
OCTOMR lt»T 

MClAL/CTmiC eiSraitUTION M0 LIMITCO.CMOLIV* MOflClCNCT STATUS Of PUPILS OISTRICT VC tUILOIMC SUMUJtlKS 

SCWOL OISTRICT COOC^ 

TMIS TAfttC MOVIMS INTOMATlON AMUT TMC KACIAL/CTHIIC OISTIIIIUTlQN AMD LIMITCO (NQLISH PKQriCICMCT ILC9I STATTJS Cf T>«C 
OtST«ICT*S nVILS fM THC PAST n«CC TCAKS. AS RCPMTKO QM THC MSIC COUCATIQNAL OATA SVSTCtt*S ItCOSl SO«0t DATA rOM 
SUmMT OATA AIIC MQVIOCO fM THC TOTAL OISTAICT AM» fOK CAO< ^OOOt IN THf OISTRICT fOR CAQ4 5OC0L TCAII T>«C TAtLC SHQmS 
THC Hjmtn or pupils aw PKRCCNT of total CMtOUMNT IN (AO< CATKOOAT. THC RACIAL/CT»«IIC PCftCCNTS HAT f«T AOO TO 100% OUC TO 
MUCINO. IAN ASTERISK IWICATCS THAT A PKRONT IS LKSS THAN 0.t.> 

••• lt«4««S •••• 

M). or % or 

llACIAl/nWIC OftlQIN MO LtP STATUS PUPILS CMtQLL. 



••• tt«S*«t •••• 

M) or % or 

PUPILS CMQLL. 



••• IMI-«7 •••• 

m Of \ Of 

PUPUS CNAOLL 



TOTAL OISTRICT 



AMRICAN INOIAM. ALASXAM. ASXM OR PACiriC ISLJMOCR 


199 


1 




t04 


9% 


I2S 


1 i\ 


suae (M)T NISPANICl 


22t 


1 


t 


:io 


1 1 


211 


1 9 


HISPANIC 


332 


2 


• 


34i 


3 0 


32t 


2 9 


NMITC IM)T NISPANICl 


tt3S0 


t4 


: 


totts 


t4 3 


tOCl7 


94 0 


LIMITCO INQLISM PROriCICNCT 


t3t 


1 


1 


i3i 


1 2 


!)• 


1 4 



SCHOOL 

AMCKICAM imtm. ALASKAN. ASIAN Oil PACiriC ISLAMOCR tt 2M% 3 t% 4 gX 

•LACK tM)T HISPANIC) 3 i tO 2 t 2 4 

HISPANIC 29 S • 23 4 7 24 4 - 

WHITC 1M)T HISPANICI 471 9t 7 434 92. S 413 94 2 

LIMITCO CPIQIISH PROriCICNCY 0 • 3 1 4 I 

Socioeconomic Indicators . Two socioeconomic indicators are 
provided in the report: 

e Socioeconomic Indicator (Census) refers to the percentage 
of children, aged 5-17, whose families are below the poverty 
level as determined by the 1980 Federal Census. This figure is 
reported for the. total district only and, since it is deterr^lned 
using information obtained in the 1980 Federal Census, the 
percentage will remain the same until the completion of the next 
census. 



e Socioeconomic Indicator (BEDS) shows the percentage of 
pupils in each school who are members of families whose primary 
means of support is a public welfare program. The numbers are 
based on data submitted to SED by the school over the past three 
years and are presented as a range, e.g., 1-10%, 11-20%, 21-30%, 
31-40%. 

Pupil Mobility Indicator . The pupil mobility indicator shows the 
percent of pupils in the highest grade of each school who were 
enrolled in that same school during the previous year. For 
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example, in an elementary school with six grades, the pupil 
mobility indicator shows the percent of sixth grade students who 
were-enrolled in the school in the fifth grade* Information is 
provided for each school building in the district and data are 
reported as a range, e.g., 1-10%, 11-20%, 21-30%, 31-40%. 

School Characteristics 

In this category, the CAR incorporates the following 
descriptive information: enrollment, attendance, and dropout 
rates; average class size; and pupil/support staff ratio. 

Enroliaent. Attendance^ and Dropout Rate s. The CAR contains 
information on enrollment and attendance by grade and dropout 
rates for grades 9-12 for the total district and for each school, 
for the past three years. 

• Enrollment rates are based on fall reports and may differ 
from enrollment numbers reported for the PEPs and PCTs, which are 
given later in the school year. 

• Attendance rates present the total number of days that 
pupils actually attended school as a percentage of the total 
number of days that school was in session. Since annual 
attendance cannot be calculated until the end of the school 
year, figures are not included for ^he current school year. 

• Dropout rates are reported for the total district and for 
each school that contains any of grades 9-12* The report shows 
the number of students in grades 9-12 whp left school and their 
percentage of the total enrollment in those grades. It is an 
annual dropout rate. The term "dropout," which is defined 
differently in other states and localities, when used by the New 
York State Education Department refers to any student who leaves 
school prior to graduation for any reason other than death and 
does not enter another school, alternative high school, 
equivalency program, or other diploma program. The number of 
students in the district who leave school but enter an 
alternative high school equivalency or other diploma program is 
also shown in the CAR report. This' number is represented as a 
percentage of the total 9-12 enrollment. 

Average Class Size . Average clans size data are presented 
for elementary grades at the kindergarten and at the common 
branches (1-6) levels. For the middle school grades (7 and 8), 
average class sizes are provided for grade 8 classes in English, 
mathematics, science, and social studies. At the high school 
level, average class size information is presented for grade 10 
classes in English, mathematics, science, and social studies. 
These grades and classes were selected to be representative of 
those with the largest .typical enrollments. 
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Pupil/Support Staff Ratio > The pupil/support staff ratio 
refers to the number of support staff personnel available to 
jprovide services to students in the total district. The ratio 
expresses this relationship by shoving the number of studentis to 
be served by each inember of the support staff. This number is 
obtained by dividing the total district enrollment in grades K-12 
by the total number of full-time staff (PTEs) classified as 
Support Staff. "Support staff" refers to guidance counselors, 
nurses / psychologists, psychiatrists, dental hygienists, 
librarians, social workers, and attendance teachers. (New York 
City^ Buffalo, and Rochester dd not include nurses and dental 
hygienists in this category.) 
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Ch :/:er 3 

LIMITATIONS OF TEE COMPREHENSIVE ASSESSMENT REPORT (CAR) 



New York State *s efforts to provide districts with a broad 
range of information constitutes an important service; however / 
its usefulness in determining the educational effectiveness of 
schools should not be overestimated. 

In this chapter we discuss some of the limitations of the 
CAR. The goal of this discussion is to provide a context in 
which districts can understand the importance of supplementing 
CAR data. 

The New York State Regents intended the CAR to provide 
information that would encourage local school districts to 
focus on improving the educational process. It would be the 
initial stimulus for districtwide assessment and planning. 
Problems appear to arise when the CAR is viewed^ not as the 
first step^ but as the final step in this process. Some 
specific limitations are enumerated below. 

Firsts the data contained in the CAR present only a narrow 
view of any district's performance. The report focuses only 
on specific instructional objectives and outcomes and describes 
outcomes using gross and limited measures of central tendency^ 
for example^ the percent of students above the statewide 
reference point. 

Assessments restricted to such statistics as "percent above 
SRP" can be very misleading. The graphs on the following page 
illustrate this point. They show a distribution of scores 
with 60 percent of the students achieving above the SRP. In 
this example the SRP is equal to a score of 69, and there were 
twenty students tested. In the top figure the test scores are 
clustered near the statewide reference point. In the middle 
figure the scores are extreme — either zero or 100. The 
bottom figure shows the data to be fairly normally distributed 
between 40 and 100. Clearly the pedagogical response to each 
of these distributions would vary. Yet the "60 percent above 
the SRP" characterization obfuscates the difference. 
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The second serious difficulty is that the CAR consistently 
presents data for different groups of students over three 
consecutive school years (e.g. third graders in 1983-84 ^ third 
graders in 1904-85^ and third graders in 1985-86). Reasonable 
assessments require, at a minimum, data for contiguous grades 
(i.e. third grade in year one, fourth grade in year two, fifth 
grade in year three) • Ideally, the same students should be 
followed over time through longitudinal analyses in order to 
develop and test hypotheses ahout student outcomes related to 
multi-year ••school effects. Examples of contiguous grade and 
longitudinal cohort analyses are contained in Appendix 1. 

A third difficulty is that CAR data are not. uniformly 
derived. Although the SED has established guidelines to^ 
facilitate uniform data collection and test administration 
procedures for all districts of the state, differing local 
policies and practices reduce the comparability of the obtained 
data. Policies y^^ry with respect to several matters: which 
students get included or excluded from components of the testing 
program; how si:udents get classified as LEP or handicapped; which 
students take practice exams; the extent to which school-provided 
data are validated; etc. Consider some specific illustrations. 

e Districts differ considerably in the numbers of limited 
English proficient students who are included in the 
test taking population. The test manual requires that 
students receiving instruction in schools in the United 
States for two years take exams in English. Districts 
that have siibstantial numbers of immigrants are at a 
marked disadvantage as the language skills of new 
speakers of English, despite instruction, will likely 
lag behind native speakers' skills « To demonstrate 
this point further, consider New York City's annual 
ranking of schools by students* performance on reading 
tests. Schools are ranked on the percent of students 
achieving above grade level. For this calculation, 
however, students who have been excused from the test 
(including those excused for limited proficiency in 
English) are counted as below grade level. 

• Handicapped students' test scores are reported 

separately from non-handicapped students' test scores 
in the CAR. While the term "handicapped" clearly 
refers only to those students who have been classified 
as such through the state-approved Committee on Special 
Education, the practice of testing and classifying 
students does, in fact, vary considerably by district. 
Some districts more effectively identify learning 
disabled youngsters than others. Consequently there 
are fewer lower abilicy youngsters' test scores 
factored into the general education data. Some 
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districts routinely wait until after the third grade 
PEP test to classify students as learning disabled. 
Consequently test scores appear to improve in the later 
grades • 

• Variation in the policy of exempting students from the 
ex2UBiinations also affects test scores. In some 
districts students do not take Preliminary Competency 
Tests (PCTs) . As a consequence scores on the actual 
RCTs may not be as high. Similarly many students take 
the RCTs in January as well as June, which tends to 
result in higher June test scores. Additionally, in 
school districts where a high proportion of the 
students take Regents Examinations rather than 
competency tests, scores on RCTs will be lower than 
expected because the exam is eschewed by more able 
students. 

Fourth and finally, record-keeping practices vary within the 
state. Procedures for collecting data are better in some schools 
and districts than others. As a result, it is difficult to make 
valid comparisons between districts. Records are less likely to 
be reliable in areas with high pupil mobility. 

Many of the limitations described above are not specific to 
the CAR but are general problems that educators must confront 
when utilizing standardized test data. In the chapter that 
follows we discuss steps analysts can take to overcome these 
problems. 
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Chapter A 

PREPARING AND PRESENTING A COMPREHENSIVE ASSESSMENT REPORT 



The New York State Comprehensive Assessment Report Is, as we 
have suggested, a useful but not sufficient tool for assessing 
school and student performance. This chapter explores steps that 
districts can take to improve upon the information made available 
by their state education departments in the preparation of a 
final local report. 

Three gerieml principles have guided us in developing these 
recommendations. First, ve believe districts would be advised to 
examine carefully the data provid&d by the state to see if the 
data are compatible with information they already have about 
themselves . 

Second, ve believe districts would benefit from 
disaggregating or separating the state's data into their 
component parts so that the data yield information about 
appropriate subgroups of students, schools, subject areas, etc. 
The New York State CAR reports information in a way that makes 
comparisons between schools, between school years (e.g. 1986 and 
1987), between academic grades (e.g. third grade performance and 
;?ixth grade performance) , and between academic disciplines 
(reading, math, writing) feasib3.e. 

Third, we recommend that when districts make comparisons 
among schoolc7, among grade levels, and among academic 
disciplines, etc., that they attempt \;o control for pre-existing 
differences in the groups being compared. At a minimum districts 
should ,attempt to find out if the students within the groups 
being compared were roughly comparable before the treatment, 
i.e., betore enrollment in the grade level, subject area, or 
school in question. 

The material that follows will elaborate these principles. 
It is the goal of this guidebook to help districts work with the 
data provided by their state education departmen'^s and move 
toward a, richer and fuller local assessment report. 

Preparing the Assessment Report 



Step One; Verifying Data 

In New York State, districts are required to sxibmit 
demographic and institutional data to the state education 
department. The state compiles this information, and it is 
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ultimately returned to schools.. In the course of these 
triBinaactions,, errors are inevitably made. Data are occasionally 
Ibst; .data are occaaionallV: inaccurately or incompletely 
recorded'; There is a clear need to check the state's information 
for accuracy and completeness. Missing data must be entered. 
Data -that are inconsistent with other sources of information on 
the same 'subject should be scrutinized to determine if they are 
erroneous or if alternative methods of calculating outcomes 
simply revealed a different .picture of the same phenomena. 

The first step in the process of verification is to gather 
together the material the district originally sent to the state 
in response to the state's request for information and check the 
transcription of that, material for accuracy. Are enrollment 
figures, class size data, racial breakdowns, etc., correct? 

Additionally, if your school district has a local, 
standardized testing program, it may wish to compile the results 
from those ^xams in. a form parallel to the state's. If the state 
reports district test data in aggregate form and grade by grade, 
attempt to organize your own data in that fashion. Compare the 
state's information carefully with your own, checking for 
inconsistencies as a way of identifying possible errors. 

To illustrate the subtlety of comparisons across tests, 
imagine the state reports 70% of your third graders at or above 
criterion on the Degrees of Reading Power (DRP) Test and that 
your own data, let us say from, the California Achievement Test 
(CAT), show 50% of third graders at or above the national median. 
One could reasonably think these data were incompatible. In the 
process, of attempting to verify the scores, you might find out 
from the state education department that the percent passing the 
statewide reference point on the DRP was generally higher than 
the .percent at >or above the national median on the California 
Achievement Test. This finding would: tend to support the notion 
that the data were accurate and coiapatible. On the other hand, 
you might find that the percent passing the DRP criterion was 
generally lower than the percent above the national norm on the 
CAT. Such a discrepancy could signal -an error, although there 
are certainly other possibilities. One would persist, in the 
latter case, until certain that the data were correctly reported. 



step Two; AnalyzinQ Trends 

After the possibility of error has been ruled out, the 
analyst's; next task is to highlight and explain inconsistencies 
in the data. If, for example, the state's assessment suggested 
there were considerably fewer children reading below level than 
the district's assessment suggested, that information must be 
reconciled. It may be that one test had a lower criterion or 
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••ipaasing*^ grade than the other. It may be that different skills 
were measured by the separate tests or that different students 
were tested* miatever the case, it is the analyst's job to point 
out inconsistencies and. attempt to reconcile all availsO^le data. 

When inconsistencies are satisfactorily reconciled, the 
process of disaggregating the data or subdividing them into their 
component parts and identifying trends begins. This is a 
painstaking task that requires considerable care and time. In 
the sectionis that follow, we will describe the process using data 
provided by the New York State Comprehensive Assessment Report. 

The CAR presents data in a manner that facilitates time-- 
ordered, within^-district comparisons between schools, academic 
levels (e^g. third grade and fourth grade) , and academic content 
areas (e.g. reading and math), as well as comparisons between an 
individual district and the state. Approaches toward these will 
be discussed. 

District Level. The threshold analytic question for the district 
is how is it doing in comparison to the rest of the state. In 
New York State, the literal emswer to that question is set by the 
percent, of district students who perform above the Statewide 
Reference Point (S^ElP) in each skill area. The CAR makes it 
possible for each district to compare itself to the statewide 
average. A more sensitive comparison, however, would consider 
how the district is doing relative to districts with comparable 
student groups; and comparaible expenditures and in comparison to 
its own past performance. This information is also available. 

After understanding how it is doing in comparison to the 
state, the district will be interested in determining if there is 
variation in its schools* effectiveness. Are students more 
likely to be successful in one district school than another? 
Again, it iis not difficult to give a literal answer to that 
ijuestion. One can simply exeunine and compare the percent of 
students above the SRP in each school. Figure 1 on the following 
page illustrates this approach to schoolr-to-school-and-district 
comparisons. In that figure third grade PEP reading performance 
is shown for a hypothetical district and for each of its six 
elementa2:y schools. 

Such a graph is easily generated from information available 
in the CAR. In school districts where the student population is 
generally homogeneous and expenditures per school are comparable, 
it is likely to provide adequate information. In more 
heterogeneous districts,, comparisons might more appropriately be 
made between, schools with comparable student populations and 
resources. ^ Figure 1 could be easily modified to cluster together 
schools with roughly comparable populations and make this 
comparison visually compelling. 
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:A next reasonable point of analysis for districts concerns 
performamce across the basic skill areas. Are students doing 
better work in reading than they are in writing? Figure 2 on the 
next pagfe shows reading and writing trend data for our 
hyjpot^etical district and for each of its middle schools. One 
cah^ flfee that it looks as if the district is doing somewhat better 
in reading • That being the case, before assuming that this 
variation in achievement is a function of variation in the 
quality of instruction, be sure to rule out the possibility that 
the difference is hot. a. result of differences in who is taking 
the tests. Perhaps the more able writers are taking the English 
Regents rather than competency exatm, and the effect springs from 
less able students being overly represented in the group taking 
the writing com^^tency test. 

Another important consideration for the district is grade- 
level performance. The CAR provides data for different groups of 
students over three years: third grade scores in 1983-84, third 
grade scores in 1984-85, and third grade scores in 1985-86. As a 
result, amalysts often focus their comparisons on separate 
cohorts, a cohort being a* grouping esteiblished by the year 
students entered :a. certain grade. The preoccupation with 
different cohorts tends to suggest that fluctuations in test 
scores are related, to variation in the quality of teaching over 
time. ' In reality the: higher test scores in the more successful 
yearsf could simply be a function of the district's having 
enrolled mpre^ advantaged children those years. 

Reasonable standards of comparison require data for 
contiguous grades, i.e. third graders in 1984-85, fourth graders 
in 1985-86, and fifth graders in 1986-87. A thoughtful inquiry 
about grade level performance might be expressed as follows: Are 
more students aibpve criterion in the fifth grade in 1987 than 
were above criterion in. the third grade in 1985? One must, 
however, be cautious here when interpreting the data. There is a 
tendency to assxime that if fewer students are above grade level 
in the fifth grade than were above grade level in the third grade 
that the school district is less instructionally effective in the 
later grades. This may be the case, but one must first rule out 
the possibility that the population shifted over time. Remember 
that new students came into the school system and others left 
over those two years, meaning that we are not precisely comparing 
the 1984-85 third grade class to itself in 1986-87, when it is a 
fifth grade class. Differences in achievement could simply be a 
function„of the changes in the population tested. Perhaps a 
disproportionately higher percentage of studentis with good test 
scores left the system over those two years, leaving a 
corresponding higher percentage of disadvantaged students to take 
the test in the fifth grade. Only when those possibilities are 
ruled out (e.g. through longitudinal analysis) can one talk 
meaningfully about educational trends. 
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Microcosm School District 
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School Levels Each school within the district would benefit from 
.a similar scrutiny of the data^ comparing their students' 
attainments: across grade levels, across time, and across skills. 
Th0' caveat regarding comparable populations still pertains when 
interpreting this information* To repeat an example given above, 
if the sixth grade in 1987 is not doing as well as it did when in 
was the third: grade in 1984, the analyst should attempt to rule 
out the possibility that the effect is largely a function of 
mobility within the population. 

To .summarize, the logic of district level analysis is framed 
by the imahner in which the data are presented by the state 
education department.. In New York State > given the _a:vailabilitY 
of.schbol and. grade level data across three time periods in 
several basia areas, it^ is appropriate, as has been 

suggested above, that. comparisons be made on all those 
dimisnsions'. The^key to such; analyses is the, attempt to factor 
out the effects of extraneous variabJiss and, to the extent 
possible, to emphasize comparisons among like groups. 

Step Three; Supplementing the Data 

During a press conference in November 1985 the Chancellor of 
New. York City Schools announced that he would develop an 
as&fessment prbces^s that wpuld go far beycnd the requirements 
placed on the schools under the Comprehensive Assessment Report. 
The state's process, he said, had resulted in the erroneous 
impression that most city schools were miseducating youngsters . 
This section describjes information New York City added to enhance 
understanding of the state's data and increase the value of those 
data for school improvement purposes. Alternatives that might be 
considered by suburban and rural areas also are included > 

Academic Outcomes. New York City school profiles include 
achievement results from locally administered standardized tests 
as well as from the state achievement tests. New York City also 
includes the promotion rates from grade to grade for its schools. 
Other school districts ;nay wish to supplement the CAR with 
criterion-referenced or diagnostic tests that can provide a 
detailed picture of content areas needing attention. Results of 
reimbursaible program evaluation testing activities also may be 
included. Districts serving substantial numbers of academically 
oriented youngsters might catalogue students • awards, 
scholarships, SAT scores, and academic honors. 

The purpose of this, additional Information is to help local 
policymakers more reliably judge if schools in their district are 
attaining their unique educational objectives. Since these are 
locally generated data, it is also possible for the district to 
report the information so as to detect underlying differences 
between the achievement of various student groups. Student 
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achievement is known to vary by students' English proficiency, 
socioeconomic status, race, and gender to name a few conditions, 
and lt< is always interesting to see if schools are having any 
succecifi reducing the predictive value of those variables* 
Althoucfh the New York State Comprehensive Assessment Report does 
not' present data in a way that makes comparisons among such 
subgroups feasible, districts can attempt to organize their own 
local information gathering so as to facilitate such comparisons. 
To illustrate how this might be done, figure 3 on the following 
page compares male and female achievement on the College Board 
with nationwide results over the past three years. 

Non^AcadeBic outcomes . While competency in reading, writing, 
and math are essential educational goals, there are many aspects 
of student growth and e.chievement th:^t are as important, although 
in cexrtairi cases, more difficult to measure-- creativity, verbal 
communication skills, perseverance ,^ irespect^ for one's own and 
other cultures, etc. The CAR does not report these student 
outcomes, but an individual district could as part of its 
assessment procedure. Accomplishments in areas such as music, 
art, commiinity service, or crafts, if they are a critical 
component of a school or district's mission, could and should be 
highlighted. New York City, not unlike other troubled urban 
areas, is particularly concerned with reducing delinquency in its 
schools. New York City, therefore, records the number of unsafe 
incidients reported in its schools* 

Institutional Characteristics . Although the quality of 
instruction is an important determinant of student success, 
research has demonstrated that there are a wide range of student 
characteristics associated with academic outcomes that are 
outside the schools' control. The New York State CAR, contains 
information about some of these areas, for example, students' 
English proficiency, economic status, race, and mobility. The 
report also includes information about institutional variables 
that are associated with variation in student achievement. These 
include such matters as school size, class size, and 
pupil/support staff ratio, etc. Other variables that play a role 
in student progress and might profitably be measured include 
teachers' years of experience, the quality of instructional 
materials, extent of athletic facilities, etc. 

In New York City, given the problem of urban overcrowding, 
the schools indicate their enrollment capacity and percent 
utilization. Suburban districts, where taxpayers directly 
determine the school budget, might have a special interest in 
specifying annual per capita expenditures for the three years in 
which there are outcome. data. (It is difficult to imagine 
a school district - sxiburban or urban - that would not be 
interested in such data.) 
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step Four ; Applying Appropriate Analytic Techniaueg 



We have paid little attention so far to the application of 
inferential statistics to the data. It is our yiev that if 
districts carefully scrutinize the state's information^ look for 
trends, and supplement, the data with local information, that they 
will, for the most part, be able to isolate educationally 
meaningful patterns without adyanced statistical techniques. 

To illustrate an idiosyncracy of significance testing, if a 
school with 30 students in grade 6 had 60% of its students aboye 
the State Reference Point on the mathematics test in 1985 and 80% 
aboye in 1986, this apparently dramatic increase would not be 
statistically significant. The difference would haye to be 
larger than 23 percentage points to be significant at the .05 
leyel of confidence. When one is working with a small group 
(N»30) , it takes a yery large difference to produce a significant 
result. 

What matters most is not whether a difference is 
statistically significant, i.e., that it is yery unlikely to haye 
occurred by chance. What matters most is whether there is enough 
general eyidence in the data and in the "surround" to suggest an 
educationally meaningful trend. 

While we are cautious about an oyer-reliance on statistics, 
there are often times when statistical techniques would be useful 
to confirm the significance of apparent trends in the data. We 
haye, therefore, included a few examples of techniques that can 
readily be applied to the data presented by the New York State 
Education Department. Those techniques are described in Appendix 
2. More sophisticated multiyariate techniques, which allow one 
to estimate and control for the effects of seyeral institutional 
and student yariables simultaneously, will enhance understanding 
still further. Among the most useful multiyariate techniques is 
multiple regression analysis, which makes it possible to estimate 
the degree to which certain factors are associated with student 
outcomes. An equation can then be deyeloped that expresses that 
relationship. Working from the equation it is possible to 
identify "outliers," or schools that do better than others in 
their category, their category being schools with similar 
concentrations of the factors that are known to accompany 
achieyement^ For example, since socioeconomic status and 
mobility are related to academic achieyement, a multiple 
regression analysis could allow us to identify schools that do 
better than expected aftei controlling for poyerty and mobility, 
i.e., schools that do better than others within their 
socioeconomic and mobility groupings. 
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We have emphasized throughout this chapter that, when 
comparing group performance, comparisons be made with attention 
to the^ equivalence of the groups being studied, A multiple 
regression analysis simply gives us a statistical way of 
establishing this control. It is a powerful technique. Figure 4 
on the next page illustrates one way in which the differences 
between schools* expected and actual scores might be presented. 
Schools scoring above the zero point have ^residuals/' or 
differences between expected and observed scores, that are higher 
than anticipated, and schools scoring below the zero point have 
residuals that are lower than would be expected. 



I>resentina the Assessment Report 



Presentation of the comprehensive assessment report can 
provide the public with an opportunity to review student 
achievement and other data, and it can be a significant step 
toward involving parents and the public in local school 
effectiveness activities. Transforming the sterile tables and 
statistics into a meaningful public presentation is, therefore, 
of utmost importance. 

Whatever, the medium employed for reporting to the public, 
clarity is essential for audience understanding, report 
credibility, and user application. In this context, clarity 
refers to explicit, unenctunbered and jargon-free narrative, 
illustrations, and descriptions. It is also characterized by 
conciseness, logical development, well-*defined technical terms, 
tabular or graphic representations, and relevant examples (Joint 
Committee on Standards for Educational Evaluation, 1981) . 

An effective comprehensive assessment report should contain: 

m a' description of the assessment process; 
m a summary of relevant demographics, so that 

an appropriate context for the angilysis can 

be framed; 

e a review of basic performance data, including 
state tests, graduation results, locally admin- 
istered tests, and other outcome indices (it is 
recommended that data presentations should develop 
from the simplest descriptive statistics 
to any of the more complex comparisons and infer- 
ential statistics) ; and 

• a summary of major findings. 
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The district's report presentation should be designed to 
stimulate, piiblic interest in collaborating on school level needs 
asseissments amd subsequent program development and evaluation 
activities. Included as Appendix 1 are excerpts from a model 
comprehensive assessment for a fictitious school district named 
Microcosm School District* 

Microcosm School District is a mid-tjiized suburban school 
district with six elementary schools, two iaiddle schools, and an 
academic high schpol* The district serves approxLnately 9,000 
students . in. grades kindergarten through twelve* In addition to 
the standard testing prograin, Microcosm analyzes College Board 
exam data and annually administers several subtests of the 
Stanford Achievement Test (grades 1-9) and the Otis-Lennon School 
^ility Test (grades 2, 4, 6, 8 and 10). 
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Appendix 1 



MODEL COMPREHENSIVE ASSESSMENT REPORT: EXECUTIVE SUMMARY 



***** Note to reader ***** 



A Model Comprehensive Assessment Report; Executive p inrnnai^- 
Tbe following illiJistration has been compiled from handouts 
presented: at December, 1986 public school board CAR 
presentations. In a typical presentation the Executiw fimsnwar y 
is presented orally by the Superintendent (or his/her designee), 
^Aiile overhead tremsparehcies* are to project the graphs. 

The following materials iure lioibuaiy distributed to the board 
members^: district and school administrators, pedagogical 
personnel, parents, paraprbfessionals, and others who attend the 
smnual CAR jpresentatioii : 

• the Executi ve giTHmar y ; 

• the state's CAR Report ? 

• the state '.i guidelines for interpreting CAR data; and 

• a locally prepeured CAR Users' Guide to aid further 
with inrr.rpretation. 



************************** 
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OCMEieiElISIVE ASSESSMENT -PEPQBT: EXEOTIIVE SUMMARY 
HICROOOSM SCHOOL DISTRICT 

TSECB/EER, 1986 



A.B. PERSCN, HI.D. 
SUEERINIENEENT OF SCHDOIS 
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Mi cr oQoe m School Distric± 
Ocnprdiensive Assessment Bi^rt 



ExBCutlve Summary 

!• I i it i rtx3 u ctic3n 

A qcoprehensl^ assessment prooess was initiated last year by the Nev York 
State Bpaxd of Begents. Ihe prooess began with the state's presentation of a 
.Ocq^xD^iiais Assessngnt R^ort (CAR) , a document that summarized state test 
results, graduation data, enrblLoent and attendanoe data, ethnic distribution, 
class size, and otb^ infonaatic^ the total district and each school over 
three years. Districts yaere periidtted to sA:^leo^^ the report with local 
ihfqn^^ relevant to the assessinent: process. 

Ihe. purpose of the Ocnprd^gisive Assessment R^rt was to provide a format 
for ihe Board, of Education, peu^its, ^nd loesstBrs of the educational conmunity 
to review. ax^d oGose^ on the findings and to encourage these groi^ to work 
together to develop plans for school inpr o v e men t. 

Ihe dcpplete state r^ort is available for review upon request. In sane 
instanoes, the CAR developed by the state reported incorrect or missing data. 
To thS' extent possible. Microcos m School District has corrected sudi data and 
si:^lemented the report with additional information. Ihis executive summary is 
designed to hi<^i^t the major findings of the CAR and to provide a concise 
description of tha district. 

This sumnary report is presented in the spirit in vAiich the state intended, 
i.e. , as a stimulus for discussion and planning and for increased involvement 
in and su^^port for t^ie educational progr a m s of .the district. It is isportant 
to raiiesober that the r^ort presents, for the most part, quantifiable 
information such as ^irollment data, attendance data, test results and 
graduation data. Such results are, sisply stated, the cutccmes of the 
interaction of many factors. Oiey should not be viewed as the ••bottcm line" 
that reflects the ability or performance level of Microoosm^s students or 
teaciiers. Ihe CAR utilizes quantifiable data to provide both a denogrE^Mc and 
performance profile of Microcosm School District. 

II. Demoaratiuc Data 

An examination of the demographic data lii the CAR shews Microcosm School 
District to be a mid-sized suburban school district. Ihere are presently six 
elementary sdiools, two middle schools, and one academic hi0i school in the 
district. During the past school year, approximately 8,575 students were 
enrolled, in district schools. This represents a sli^t decline from the 9,047 
students enrolled in the 1984-85 school year. Ficrure 1 shews elementary school 
enrollments for the district, as well as for each of the district's six 
elementary sdiools. It can be seen in the figure that, ^Adle there was an 
overall decline in enrollments, patterns varied fixm school to school. 
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Elementary School Enrollments 
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Htm aamual attendance rate reportecl for the 1984-1935 school year for 
MicxoqoiB ff^i^T District is 93%. This rate has reanninpri stable since the 
1982-1983' adibol year. T3m drc^^out xate/^ch is the peroentdge of pupils vho 
hav«i left school four any reascn except death or to enter another school, had 
ihcTiaMd ioig^ fron 3.8% in 1983-84 to 4.3% in 1984-85. Attendance and 
dtqpeixt^data for;-^ 1985-1986 school year indicate that attendance has 
maihad at about 93%; and that the percentages of students vft)o dropped cut of 
school or iiitp have entered an alternative high school diplcna or other diploma 
progpm have declined slightly. 

With regard to ethnic distributicn,. Microcosm School District is primarily 
\4iite (94.3%) , with small percentages of students frcm Hi^»nic (3.0%), black 
(1.8%),. and, American Indian, Alaskan, Asian or Pacific Islander (.9%) 
backgrcunds. i^;$ax&dBately 1.2% of the students enrolled are limited English 
proficient, i.e., they, by reascn of foreign birth or ancestry, speak a 
language other than. Qiglish and vmderstand or speak little or no English, ihe 
GAR also reports socioebononic indicators for the region. Ihe 1980 Federal 
Census indicated that appoExadmately 7% of the students in the district are 
nen b e rs of families below the pcMuty level. 

Ihe Ocnprehensive Assessment R^ort presents class size information for 
classes in vhich enrpUnKit is generally the largest: kindergarten, grades 
1<^, grade 8 and grade 10 crre subjects (English, math, science, and social 
studies) . Hialooo6m*s average class size> across all grades, is approodnately 
25 students. £^ classrocn teacher is, of course, sij^ported by other subject 
area teachers, .suc^ as those vdio provide instruction in lojsic, art, physical 
education, octputer education, vocational education and library. A local study 
of the. actual number of teachers in the. district and tiie total pipil:teacber 
r&:'/.-> supplesaenirs the average class size data presented in the CAR. Over the 
pass: three years, Hicrocosa School District has provided pupil: teacher ratios 
of IS stiadents to one' teacher at the elementary school level and 15 students to 
one teacher at the sebondary school level. Ihese ratios reflect the 
abovto-rffioitioned sapgast staff. 

togpe^ fera will ba interested to knew that, over the past several years, the 
tax rata has risen at a auch skewer rate than the district's ocsating 
^igffienflea. ' Tha argaxved ooegatina expense of th e district, as represented by a 
c ost par PUPil. i ose froa S3.142/PUPil in 1982-83 to $4046/txipU in 1985-86. 
a his rsp y eaents arrinerease of 28.8%. Despite thes e increasea. the tax rate 
has increased bv only 6.2% in the same period - fron $45.64 per SIOO/AV in 
1982-83 to S48.48 pe r lOOAV during 1985-86. 



38 



48 



Htm Ouui ^ f im)H ivm Assessaent Bepoi± focuses on quantifiable performance data 
such' as state test rasults arid hi^ school graduation results in order to 
wffWMS the district's past perfonDance and c u rr ent status. Last year the State 
Bducatiioh Dip attittBii b examined such data for over 6000 sdiools in the state and 
identified the 600 schools with the poorest results in these areas. Sudi 
Khools nust disyelcp and isplenent plans for isproving their performance. 

Mot surprisinglYr no schools in Microcosm School District were identified 
ancm those with tiie txxprest results. The review of performa nce data for 
Microdosm indicates generally h igh levels of student achievement. Ihe CAR 
:dtan/elqpiBd fay the state is suppleaoaented in this sunnary with locally 
administered abiiit/ and acMevanent test data, including the Otis-Lennon 
Sdibol Ability 1\sst, the Stanford Achievement Test and the Oollege Board 
examinations. The review of these data enable the district ^'^"rter to 
pridritize needs and plan for p rogr a m iiqprovement. Ihe analysis begun last 
year, >A)en..the'CAR r^xsrt provided achievement results for three years 
beginning m 1982-1983,^ has been extended into the 1985-86 s(±ool year. Die 
four-year trends are sumnarized below. 

A. State Tests 

Test results are presented for four state testing programs; the Pupil 
Evaluation Program (PEP) , Preliminary Occpstency Tests (PCDs) , Regents 
Ccnpetency Tests (PCTs) , and Regents ecami nations. 

1. Pupil Evaluation Program (PEP) 

TbB Rpil Evaluation Program assesses reading and mathematics in grades 3 
and 6 and writing in grade 5. PEP tests are provided for use in the early 
identification of students vdio need special help in developing the basic skills 
of reading conprehensiony mathematics, and writing. 

Figure 2 shews four-year districtwide trend data on the PEP series for 
grades* three, five and six. Figure 2a shows the same data for PS 01. 
Transparencies of Figures 2 and 2a are used to conpare the school to the 
district's performance. By placing PS 01*s transparency over the district's 
transparency it is easy to see discr^>ancies. 

Figure 3 illustrates another approach to making school-to-district 
cccparisons. In this figure third grade PEP reading performance is shewn for 
the district and for each of its six elementary schools. 

Across the four year period^ Microcosm School District has demonstrated 
gains in the percent of students above the State Reference Point (SRP) in these 
skin areas. At the third grade levels 90% of the studttnts wera above the SRP 
in reading in 1985-86^ >4iereas only 78% were above the SRP in 1982-83. Similar 
progress was made in third grade math^ with the percent of students above the 
SRP increasir^ frcm 77% in 1982-83 to 98% in 1985-86. 
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At tbB sixth grade level, the percent of students above the SEP in reading 
increasedl ftaa 7^^ in 1982-^3 to 86% in 1985*-86; for math, the percent 
ihc3?eas6d &cni 78% in 1982-^3 to 93% in 1985*86. Small declines in sixth grade 
readihg and 1984*-8S to 1985--86 are not statistically significant. In 

tbB^sreai oi writing, the percent of the fifth grade students above the SBP 
increased frcn 73% in 1982-83 to 95% in 1985-86. 

2. Preliminary Oocyetency Tests (PCT) 

Urn Preliminary daxpetency Tests in reading and writing are given in either 
grade 8 or grade 9. liJoe the CEP tests, their primary purpose is the 
idexidLf iiaticn of stxdents who haed qpecial help in developing the basic skills 
of reading cGn^r^^ensicn auid writing. Across the four-year period, Microcosm 
Scbod District has d n i i t i is Ua ted excqptiGnal nerfbrnanbe in the area of 
reading, with' 96% of the students above the SRP. In writing, tiie percent of 
stuKlehts above the SRP has increased frosi 85% in 1982-83 to 91% in 1985-86. It 
should be zbt^ d^; th^ ipaist four years, PCT reading perfomance at 
Middl^. School B has been' Shoving a steady non-significant decline, ^diile FCr 
writii^ adkevaoaar^ at the sane school has had a dramatic and significant rise. 
Fioure^^ 4 shows PCT reading ard writing trend data for the district and for eadi 
of its middle sdibols. 

3. Recrents CcitpetexTcy Tests (RCT) 

In order to receive a local hic^ sdiool diplona^ students wast danonstrate 
cccpetehce in reading, writing and mathematics. Ihe Begents Ocnpetency Tests 
(BCTS) pixvide one means for demonstrating the required level of proficiency. 
In Jost schools throu^iout the state, the RCTs are taken only by those students 
v430 are riot planning^ to attend college and ^ do not take Regents 
©caminatidns. Most coUege-bpund students satisfy the ccnpetency requirements 
by passing Pegents examinations.. 

Acaross the four-year period, the nuisber of students in Microcosm schools 
taking Regents Ocopetency Tests in January and JUne in all skills areas has 
decreased. Because tiie enrollment remained relatively stable during the fcur* 
year period, we may infer that the decrease in the number of students takii^ 
RCTs was due to an iirrease in the nuittoer of students satisfying the ccnpeteiKy 
reguiresents by pa s g ing Regents eseaminations. Ihe data also indicate that the 
percent of students passing the January RCT in math increased frcm 51% in 
1982-83 to 59% in 1985-86. There was little diange in the percent of students 
passing the January RCT in reading^ with 95% passing in 1982-83 and 94% passing 
in 1985-86. Ihe percent of students passing the January RTT in writing 
increased fxcm 79% in 1982-83 to 82% in 1985-86. 

For RCTs administered in June^ the percent of students passing remained 
stable in math (69%) . Ihe percent of students passing the RCT in reading 
fluctuated frcm 90% in 1982-83 to 31% in 1983-84 to 81% in 1984-85 to 78% in 
1985-86. Ihese fluctuations may be due to the small number of students taking 
these exass (N < 42) . In the area of writing, the percent of students passing 
the RCT increased dramatically over the four-year period frcm 50% in 1982-83 to 
86% in 1985-86. 
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NYS Preliminary Competency Tests 

Microcosm School District 



100 



■7- 



1.M 



i 

m 



m 



1:^ 



RDG:A 



ill 



RDG:B 



K 

/ 



1 




WTG:LEA 




WrG:A 




il 

"-mi 



WTG'.B 



District and Building Performance 
CZiD 83-84 K22 84-85 



85-86 



F i <i u 1 o 4 



4* Regents Exaitdj^tions 



Pegertts examinations eore achievement tests based on the oourses of stucfy 
reoGraoenctecl the State Ectucation Department for use in grades Begents 
exsoidnatlGns^^ students, parents, oqunselors, teachers, administrators, 

college d^adsaiana officers and enplqyers with objective and easily understood 
achievement informatioh. Passing scores en the Regents examinations in Snglish 
and mathematics satisl^ ^ the ccnpc^tency testing requirements for a hi^ 
school dLpioDa. Figure 5 illustraites districtwide trend data for performance 
on several pairs of Regents examinations. 

Ihe CAR^ presents R&gents ebcaminations results for January and Jlme 
administraticn:. periods . In general, few students todc exams during the January 
examination \p^ For the June administration period, it is interesting to 
^caanwe three pairs of tests that are typically taken as pairs by the same 
s1n,vterrts; English and Social Stuiies, Biology and Math 10, Chemistry and Math 
11. Olie data indicate that performance in biology, chemistry, and mathematics 
have , generally declined- in the past four years • Ihe declines in biology and 
matheanatics agyear to be statistically significant* Ihe percent of students 
passing Engli^ vas always hi^ier than the percent passing Social Studies; the 
percent passdjig Biology v;as always hi^ier than the percent passing Hath 10 (a 
ne^r math Regi^ts - Sequential Math II - replaced Math 10 and was administered 
in 1985-86) ; and the percent passing Chaiiistiy was always higher than the 
percent passing Math 11. While there were fluctuations^ in the percent passin? 
the^Regents examinations in these areas across fcur year period, it is 
inportant to note that the number and percent of students taking Regents 
examinations generally increaised during the period* Across New York State, 
2?5)raximately 60% of the students in grades 9-i2 take at least one Regents each 
year, in Microcosm^ the percent taking at least one Regents each year 
increased fcom 59% in 1982-83 to 68% in 1984-85. This represents a 15% 
increase in the- number of Microcosm School District students taking Regents, 
This datum was not provided by the State Education Department for 1985-86. 

B. Graduation Data 

The OR provides information about the district's students v4io were 
ca xiidates for graduation during each of the past three years. Fixture 6 
d^icts Microcosm's hi^ sdiool graduation results relative to performance 
statewide. In the typical higSh sc1kx)1 in New York State, apprcxiBately 90% of 
the diplana candidates receive a local hi^ school diploma and s^roximately 
40% receive a Regents diplana. For students vAio fail to graduate, the primary 
reason is failure to satisfy a local course requirement. In Microcosm, the 
percOTt of local hi^ school diplanas has remined stable over the past four 
years - 96.8% in 1982-83 and 96.6% in 1985-86. Students in Microcosm 
demonstrate a hi^ier graduation rate than those in New York State as a whole. 
It should be noted that there has been a steady decline in the percent of 
students v4io receive Regents diplcinas in Microcosm, from 44.9% in 1982-83 to 
35.8% in 1985-86 (4.2 percentage points) . ^ Tti^s decline will be discussed at 
upccmxng district- and school-level planning meetings. 
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4v Regents Exaniinatic3ns 



'Begerxt^ cbezoaiiations are achleveroerit tests based on the oourses of study 
readBwoeridB^ the State Education Department for xise in grades 9-12. Pegents 
eocasdjiatidra ^E^^ parents, counselors, teachers, administrators, 

ooHege admissions officers and eDplnyers with objective and easily \2nderstood 
adiie^^eane^^ infomaticn,. Passing scores on the Pegents examinations in English 
and^sathemtics satisfy all t±e coipetency testing requiresients for a high 
school dJplana. Figure 5 illustrates districtwide trend data for performance 
on several pairs of Regents examinations. 

!Ihe CAR preserfts Pegents examinations results for January and Jtme 
administration periods. In general, few students took exams during the January 
examination pecidd. For the ; Jujie adminisicration period, it is interesting to 
exandjie three pairs of tests that are t^ically ta]cen as pairs by the same 
students:. English and Social Studies, Biology and Math 10, Chemistry and Math 
U. Ihe data i^ perforaemcs in biology, chemistry, and mathematics 

have geneo^lly declined in the past four years. Ihe declines in biology 'and 
matheosatics; ^ffpear to be statistically significant* Ihe percent of students 
passing Engli^ was always hi^ier than the percent passing Social Studies; the 
percent :pasising Bioltsgy was always higher than the percent passing ^Math 10 (a 
new math Regents ^ Sequential Math II - replaced Math 10 and was administered 
in 1985-*86) .; and the percent: passing Chemistry was always hi^ier than the 
percent passing Mat^ 11. ^fhile there were fluctuations in the percent passrjig 
the Pegents examinations in these areas across the four y^or period, it is 
isportaht to note that the number and percent of students taking Pegents 
examimtions generally increased during the period. Across New York State, 
egprgximately 60% of the, students in grades 9«*12 take at least one Parents each 
yeari In Microcogm, the percent taking at least one Regents each year 
increased from 59% in 1982-83 to 68% in 1984-85. This represents a 15% 
increase in'the number of Microcosm School District students ta3dnq Recrents, 
This datum was not provided by the State Education Department for 1985-86. 

B. Graduation Data 

The CAR provides information about the district's students who wei^ 
candidatas for graduation during each of the past three years. Figure 6 
depicts Microcosm's hi^ school graduation results relative to performance 
statewide. In the typical hi^ school in New York State, a^^jroximately 90% of 
the diplcna candidates receive a local hi^ school diploma and appraximateiy 
40% receive a Regents diploma. For students ^fit)o fail to grsiduate, the primary 
reason is fadlxare to satisfy a local course requirement. In Microcosm, the 
perdent of local hi^ school diplcmas has remained stable over the past four 
years - 96.8% in 1982-83 and 96.6% in 1985-86. Students in Microcosm 
demonstrate a hic^ier graduation rate than those in New York State as a vAiole. 
It should be noted that there has been a steady decline in the percent of 
students v*io receive Regents diplomas in Microcosm, frm 44.9% in 1982-83 to 
35.8% in 1985-86 (4.2 percentage points) . This decline will be discussed at 
upcaning district- and school-level planning meetings. 
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4* Recrents Examinations 



Pegents ;«X2aiinati are achieveiQent tests based on the ocurses of study 
recxsnended by the State Education D^^artsent for use in grades 9-12. R^ents 
exazoinaticns provide students, parents, oounscdors, teaciiers, administrators, 
pqllege admissions ^officers and esployers !*dth objec;±lve and easily imderstood 
achievaoent information. Passing scores cn the Kegents examinations in English 
and mathematics satisfy all the cccpetency testing reguireinents for a hl^ 
sctabpl ^diplocia. Figure 5 illustrates districtwide trend data for performance 
on several pairs of Regents examinations^ 

The- presents Regents examinations results for January arid June 
adm i n i s tration periods. In gex^eral, few students .took exams during the January 
effloninatidn peridd. For the June administration period, it is interestir^j to 
examine three padrs of tests that are typically taken as pairs by the same 
studtents: English and Social Studies, Biology and Math 10, caiemistry and Math 
11. The data indicate that performance in biology, diemistry, 'and mathematics 
have generally declined in the past four years. Ihe declines in biology and 
nathematics a^jpear to be statistically significant. The percent of sttvients 
passing E^li^ was always hi^ier than the percent passing Social Studies; the 
percent passing Biology was alvays lii^ier than the percent passing Math 10 (a 
new math Regents. -* Sequential Math II - replaced Math 10 and was administered 
in 1985-86) ; and the percent passing Chemistry was always higher than the 
percent parsing Math^ 11. VJhile there were nvictuations in the percent passii^ 
the^ R0g^±s examinations in these areas decrees the four year period, it is 
iii|»i±ant to note that the nuzhber and percent of stixtents taking Kesgents 
examinations generally increased during the period. Aoross New York State, 
approximately 60% of the students in grades 9-12 take at least one Regents each 
year* Iii Microcosm, the percent ta3dncf at least one Regents each year 
incgreased from 59% in 1982-83 to 68% in 1984-85. This represents a 15% 
Increase in the nuntoer of Microcosm School District students taldncf Racrents^ 
This datum was not provided by the State Education Department for 1985-86. 

B. Graduation Data 

The CAR provides information about the district's students \A)d were 
candidates for graduation during each of the past three years. Figure 6 
d^i.cts Microcosm's hi^ school graduation results relative to performance 
statewide. In the typical hi^ sdbool in New York State, approximately 90% of 
the diplcna candidates receive a local hi^ sdiool diploma and approximately 
40% receive a Regents diploma. For students ^fit)o fail to graduate, the primary 
reason is failure to satisfy a locad course requirement. In Microcosm, the 
percent of local hi^ sdiool diplomas has remained stable over the past four 
years - 96 •8% in 1982-83 and 96.6% in 1985-86. Stxrients in Microcosm 
deroonstrate a hi^ier graduation rate than those in New York State as a whole. 
It should be noted that there has been a steady decline in the percent of 
students who receive Regents diplomas in Microcosm, frcm 44.9% in 1982-83 to 
35.8% in 1985-86 (4.2 percentage points) . This decline will be discussed at 
iqpccming district- and school-level planning meetings. 
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C.. Dxally A3ministered Standardized Tests 



Zh adiition to administering the state tests, Microoosm School District also 
administers an extensive standardized adiievement and aptitude testing 
p r og ra m^ "Die Stanford Adiievement Test is administered to students in grades 
1-^. Oie Otis-Lennon School Ability Test is adndnistered to students in grades 
2, A, 6, 8 and 10. The College Board exams are administered to college-bound 
seni.Qrs. An examination of results frcm these testing programs provides 
additional information about the district. 

1. Oollecre Board Exams 

The Oollege Board exams measure the scholastic altitude of college-^xund 
seniors in verbal and mathematical skill areas. In the verbal area, students 
nationwide showed iupxfvements over the four-year period; average verbal scores 
rose sli^tly from 425 in 1982-83 to 431 in 1985-86. Oollege^DOund seniors in 
Microcosm demons tr ated ^uctuations in verbal scores across the same period, 
with their average verbal score varying frcm 418 in 1982-83 to 427 in 1983-84 
to 417 in 1984-85 to 422 in 1985-86. 

In the mathematical airea, students nationwide also showed Improvements 
across the fcur-year period; average math scores rose slic^itly frcm 468 in 
1982-83 to 475 in 1985-86. OollegeHxund seniors in Microoosm demonstrated 
jQuctuation in math scores across the same period, with their average math 
score varying frcm 480 in 1982-83 to 475 in 1983-84 to 461 in 1984-85 to 460 in 
1985-86. Oollege Board mathematics data are ill\:istrated in Figure 7 . 

Overall, the average performance of Microcosm School District students 
parallels that of students nationwide, with sli^t variations by sex across the 
years. (Hiere is a sli^t decline among the boys and steady, but sli^tly lower 
performance among the girls.) 

Despite a sli^t decline in the number of enrolled seniors, the number and 
percent of students taking the College Boards at Microcosm has increased over 
the fcur-year period. 

2. Stanford Achievement Tests 

The Stanford Achievement Tests measure student achievement in a wide range 
of skills, inclining reading, mathematics, and language. Ihe Stanford 
Achievement Tests measure students' achievement in ccccparison to a 
representative nationed sample or norm <^roap, and are thus referred to as 
norm-referenced tests. Norm-referenoed achievement tests enable educators to 
ccopare the performance of an individual student or group of students to that 
of students in the norm group sanple, v*iich was selected to be r^resentative 
of all major student populations in the country. 
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!Ihe -Stanford Achievem^ Tests indicate the average percentile rank of 
Micgoodsa students asoong their peers naticrMide. An examination of Microcosm's 
perf onnanoe on the Stanford Achievement Tests in Reading and Math (grades 3 and 
6) and in language (grade 5) is provided in order to supplesnent the Pi^il 
Evaluation Krogram data presented previously. 

Qne^vay of examining performance on the Stanford is in terms of grade 
equivalent (G.E.) scores. Grade equivalents measure growth in reference to 
grade norms that are established by testing, re p resentative grcxps in each of a 
series of school grades. If students are tasted in May (the 8th month of the 
school year) , the expected grade equivalent score would be 3.8 for third grade^ 
4.8 for fourth grade, 5.8 for fifth grade, etc. Ihe ea^gcted G.E. represents 
the mean of the national norm group. A G.E. score hi^ier than the mean 
G.E. score for the grade indicates performance that is above that expected for 
a student at that grade level; a lower G.E. score indicates below average 
performance at that r;rade level. 

In the area of reading. Microcosm students showed generally rising 
G.E. scores in Total Reading, with the average G.E. rising from 4.6 in 1982-83 
to 5.1 in 1985-86 for grade three, and frcm 8^3 in 1982-83 to 9.6 in 1985-86 
for grade six. Ihe es^^ected G.E. was 3.8 for grade 3 and 6.8 for grade 6. At 
both grade levels, mean acMevement was far above the expected level of 
performance for the grade. Figure 8 illustrates Stanford Achievement Test 
results for the .various reading subtests (word study skills, cccprehension, and 
vocabulary) , as well as for total reading. 

A similar pattern was shewn in math, with the ?. rerage G.E. scores increasing 
most years for both grad£i 3 and grade 6 in every ubtest and in Total Hath. In 
the area of language, increased mean G.E. scores ere shc^rn eac^ year for grade 
5 in Total langmge. 

Another way of looking at Stanford Adiievement "^st performance is in terms 
of per<»ntile (%ile) ranks. A percentile rank indicites the percent of scores 
exceeded by the individual or group scoring at that rank. For exaiiple, a 
percentile rank of 85 means that students ^*o attain the score corresponding to 
that rank exceed the scores of 85% of the students in the national norm group. 
T!tiB mean of the national norm group is the 50th percentile. 

With the exc^Jtion of third grade vocabulary, in the area of reading both 
third and sixth grades demonstrated increased average percentile ranks on every 
reading subtest. For exanple, mean ccnprehension scores incareased for grade 3 
frcm the 89th percentile in 1982-83 to the 95th percentile in 1985-86; for 
grade 6, frcm the 91st percentile in 1982-83 to th^ 96th percentile in 
1985-86. 

A similar pattern was shewn for math, with increases since 1982-83 in the 
mean percentile rank for both grade 3 and grade 6 on every subtest. For 
exanple, mean applications scores increased for grade 3 frcm the 87th 
percentile in 1982-83 to the 96th percentile in 1985-86; for grade 6, frcm the 
85th percentile in 1982-83 to the 97th percentile in 1985-86. 
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Par all language subtests, grpdB 5 students incteased their percentile rank 
each year, with the average Total language score rising frcm the 81st 
percentile in 1982-83 to the 94th percentile in 1985-86. 

Stanford Achieveanent Test data edso are used by Microcos m to evaluate the 
district^^ state-funded BSEN (Pupils with Special Educational Nteds) diagnostic 
readijng pr o g r am . .All participating students yaere pretested and p u bL Lested on . 
e^^propriate levels of alternate forms of the Stanford Achievement Test in 
Beading. U.S. Office of Biucation Model Al was used to analyze the data. A 
control grcup is not required in this design because the onuvtrison is made 
between' t^ student's expected p ro g r e ss without t±eatment (pretest normal curve 
ecjcdvale^ and his or her actual progress with t^reatsnent (posttest normal 
curve equivalent) . Mean differences between pre* and posttest administrations 
were tested for statistical significance. Pesults indicate: significant gains 
in grades 4, 5, 7, 8 and 9; small but nonrsignificant gains in grades 1 throu^ 
3 ; and a significant decline in grade six. Figure 9 illustrates these 
findings. 

Finally, Stanford Achievement Test results are used by Microcosm in more 
sophisticated analyses of school effectiveness (i.e. , multiple regression 
analyses) . These analyses attenpt to identi^ which >s(±ools are most effective 
in; reading cdven the set of conditions that aire known to impact xipon 
acfaievenent e>q., poverty, mobility^ attendance, pupil/teacher ratio> 
teadiers* experience etc. Effectively done they enable us to say Which of 
Microcosm's schools are doing better (or worse) than exp&ctoed, based on several 
of the sdiools* significant presenting conditions. Performance patterns for 
the past four years shcM SS 1 to be performing above es^ectation vMle PS 5*s 
performance is belcw expectation. Ihese data are shown in Figure 10 . 
Inplications of these findings are being reviewed with eadi school*s planning 
team. 

3. OtiS'-Lennon School Ability Test 

Bie Otis-Lennon Sdiool Abilily Test (OISAT) is a power measure of abstract 
thihkii^ and reasoning abilily designed to predict success in cognitive, 
sc±ool*-related activities. Ihe OISAT provides scores in terms of a School 
Ability Index (SAI: a measure of sdiool ability similar to the IQ) and 
percentiles. An examination of Microcosm School District's OISAT data 
indicates that^ en the average, students in the district demonstrate school 
ability indices that are approximately one-half of one standard deviation above 
the norm. Ihis means that Microcosm students are at the hi^ end of the 
"average" abilily range. Ihe percentile ranks of the OISAT serve as predictor s 
of sdiool performanoe. Using OISAT percentiles, the predicted performance of 
Microcosm students would fall in the 62nd-79th percentile range. It is 
important to note that their actual performance is far hicdier than their 
predicted performance* This means that students in Microcosm Sdiool District 
are achievincr at levels hicfter than would be expected frari their OISAT scores^ 
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17* Qanclusicai 



The data presented in the caR and sijsplenented by the district shew 
Microoosm generally to be performing at a hi^ier level no^r than four years^ago 

• student perfomanoe has increased -on state tests; 

m more students are taking Begents exams; however there 
are significant declines in the percent of students 
passing biology and mathematics Pegents and 
proportianately fewer graduates are receiving Pegents 

' diplonas; 

• achievement* of Microcosm students in ccnparison with 
their peers in a national norm gracp is hi^ier than their 
es^ected grade levels, and ctverage performance exceeds 
that of at least 90 percent of their peers nationwide; 

m performance has increased steadily over the past four 

years on standardized, norm-referenced adiievement tests; and 

m reimbursable reading program (FSEbT) participants in five 
of the nine grades demonstrate significant grcwth in 
reading, three grad£^ show slight improvement, and one 
grade shows a significant decline. 



Ihese findings demonstrate that the district is on solid educational 
ground. Eadi school will utilize the data in the Cccprehensive Assessment 
Report to: examine its past performance and current status; identify and 
prioritize need; and plan for school iaprovement. The district welcomes the 
continued involvement and support of the Board of Education, parents, and the 
ccraraunily in planning programs that will effectively and efficiently address 
the needs, abilities and interests of all sti:dents in Microcosm School 
District. 
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Appendix 2 

B ^SIC MEASUREMENT CONCEPTS AND ISSUES 

The piirpose of this appendix is to introduce some of the 
most basic measurement concepts and statistical techniques used 
in assessing school effectiveness and school change • 

Understanding Noraative Test Data 

Test Construction 

. Can we use standardized tests to evaluate schools? Can we 
specifically use them to determine if schools are improving their 
capacity to educate youngsters over time? There is no completely 
satisfactory answer to that question. Understanding the dilemma 
requires a close examination of the basic attributes of tests ; 
especially validity and reliability. 

Validity . The most important characteristic of a test is its 
validity, which refers to the capacity of the test to measure 
what we want it to measure. There are three basic types of 
validity: (1) content validity, which measures the degree to 
which a test covers knowledge involved in a particular course of 
study ; (2) criterion-related or empirical validity , which 
indicates how well a test predicts an individual's behavior in a 
specified situation (it is usually obtained through a correlation 
coefficient that expresses the tendency for values of the test to 
change systematically with the values of the criterion), and (3) 
construct validity , which refers to the extent to which a test 
may be said to measure generalized traits or theoretical 
constructs like intelligence and neuroticism that are presumed to 
inhere in individuals over time. 

Reli2d)ilitY> Test reliability is necessary (but not sufficient) 
for good validity. By test reliability, we refer to the extent 
to which a test will rank order scores similarly for a group of 
people under differing conditions or situations. Note that it is 
reproducing the rank, not the same score, that is relevant. If 
everyone *s scores were to change by. •the same amount under two 
conditions, the reliability of a test would be perfect. 

Viewed from the perspective of reliability and validity, the 
central intellectual problem in test construction is to derive 
instruments in which the expected value of the indicators varies 
directly with variation in the construct or criterion and the 
rank ordering of individuals remains consistent. Standardized 
tests are not particularly effective instruments for measuring 
learning or change because they are constructed to yield stable 
rank orderings over time. Given this objective, items are 
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selected for standardized tests that best differentiate pupils. 
Items that measure skills that are effectively taught to all 
students In school are not likely to be Included In a test of 
that skill since they would have very little discriminatory 
power • 

The c[uostion of using these instrtiments to mear^ure growth, 
the: >fore, evolves into a discussion of whether it is possible to 
infer learning or cognitive growth from changes in students • 
relative positions on standardized tests despite the fact that 
the tests were not designed for that purpose. Understanding the 
issues involved requires an examination of the measurement 
properties of tests. 

Levels of Meaaurement 

The .most critical aspect of standardized achievement tests 
that must be considered when using these instruments to assess 
learning regards the levels or scales of measurement and the 
interval properties of the scales. A scale is a scheme for the 
numerical representation of the >ralues of a variable. There are 
four types of scales; nominal, ordinal, interval ^ and ratio. 

Nominal Measurement, Many objects have characteristics that 
differ in kind only. Religion, occupation, and gender are 
examples of such variables — one is, for example, either male or 
female. In nominal level measurement, each value serves merely 
as a label or name. No assumptions are made about relations 
between the values assigned to the data. 

Ordinal Measurement. When it is possible to rank categories 
according to some criterion, the ordinal level of measurement is 
applied. Ordinal measurement does ir^ply a hierarchical ordering 
of values; however, the distance between any two values in an 
ordinal scale is unknown. It is therefore accurate to say that a 
score of 5 is higher than a score of 4, but how much higher is 
unknown. Additionally, the distance between 4 and 5 is not 
necessarily equal to the distance between 9 and 10. For example, 
the classification of enlisted personnel provides an ordering 
according to military rank, and each category has a clear 
position in the hierarchy. It should not be implied from this 
ordering that the difference between one rank and another is 
equivalent; distance is ^lot measured. 

Inteirval Measurement, With interval measurements the distance 
between values is meaningful. Unlike ordinal measurement, 
interval level data require the distance between one and uwo to 
be equal to the distance between 95 and 100, or any other set of 
scores. It is this property of interval scale data that permits 
one to add and subtract She values. The classic example of whis 
type of scale is the thermometer, which measures temperature in 
degrees that are the same distance from each other at any point 
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In the scale. The difference between 30 degrees and 31 degrees 
Is the same as the difference between 8 degrees and 9 degrees. 
While an interval scale allows us to study differences ^ we cannot 
use it to discuss the proportionate magnitudes of those 
differences because an interval scale does not have an inherently 
determined zero point. We cannot say^ therefore^ that a 
temperature of 60 degrees is twice as hot as a temperature of 30 
degrees • 

Ratio Moasurement. A zero p^dnt car* be meaningfully designated 
with ratio measurements. It is, for example, possible to say 
that a car has traveled zero miles. Given an inherently 
determined zero point, it is possible to say that one car has 
traveled twice as many miles as another. 

The meaning of these scales of measur^ent for the 
interpretation of test data is critical. Without interval 
measurement, growth rates in time or across groups of students 
cannot be measured. Some wou3 d say that because achievement 
tests are constructed to prov-'^e ordinal relationships, i.e., 
rank orderings of individuals, it is not strictly possible to 
sxibtract scores and arrive at a s.tatement of differences that can 
be interpreted, as learning. In response to this limitation, test 
construction experts have scaled test data to yield intervals. 
The rationale for imputing intervals is that the true values of 
the underlying construct are assumed to be normally distributed. 
Given interval level measurement, tests can be used to measure 
change across time since sxibtraction of s::ores is permitted* The 
discussion below provides some insights into these transfor- 
mations of test data into equal interval data. 

Score Types 

. The problems using test scores to infer change or cognitive 
growth are not entirely resolved by the transformation of ordinal 
data into interval data. Each of the resulting score types has 
limitations. Those limitations are discussed below. 

Raw Scores, The raw score is the niimber of questions answered 
correctly. For most users of test data, the raw score is of 
little value and must be translated into converted or derived 
scores. 

Scaled Scores > Scaled scores express the results for a 
particular test and all forms and all battery levels - preprimer 
to advanced. While the scores allow one to see change over time, 
they have no inherent meaning or interpretive value. Percentile 
ranks, stanines, normal curve ^equivalents and, to a lesser 
extent, gjE'ade equivalent scores are preferable. 

Percentile Ranks, A percentile rank indicates the relative 
standing of a pupil in comparison to the norm group. Percentile 
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ranks range from a low of 1 to a high of 99 with 50 indicating 
typical performance. By way of example, if a pupil obtains a 
rank of 25 in reading, this means that 25% of the pupils in the 
norm group at the pupil's grade scored as well as or lower than 
the pupil; 75% scored higher. An important caution to keep in 
mind is that percentile scores tend to ••bunch up" in the middle 
of the scale and spread out toward the ends, meaning that an 
increase in raw score points might translate into 20 percentile 
rank points in the middle of the scale and only 2 percentile rank 
points at either end of the scale. 

Stanines> A stanine is a value on a nine point scale. There 
is a definite relationship between stanines and percentile ranks, 
with each stanine' takinc in a certain percentile band. Stanines, 
however, are equally spaced steps, and they do not bunch up or 
spread out at different points as percentile ranks do. The 
relationship between stanines and percentiles is illustrated 
below. 




stanines 



aelow4 I 4-10 1 11-22 1 23.39 1 40-S9 1 60-76 1 77-83 1 89-95 I Abov< 95 
Percentile Ranks 



Below Average 



Average 



Above Average 



Broad Classifications 



A. Normal Distribution of Stanines, Percentile 
Ranks, and Performancu Classifications 



Grade E quivalents > A grade equivalent indicates the theoretical 
grade placement of pupils for whom a given score is typical. If, 
for example^ a raw score of 35 corresponds to a grade equivalent 
of 4.9 it is interpreted that a score of 35 is typical for 
students in the ninth month of thA fourth grade. Grade 
equivalents are losing their popularity ^ in part because lay 
persons frequently think that a pupil has mastered the curriculum 
of the grade corresponding to the grade equivalent obtained. 
When, for example, a student in the third grade obtains a grade 
equivalent of 5.4 in math, this is often misinterpreted to mean 
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tnat the pupil can do the work taught in fifth grade, it is 
possible for a pupil to obtain a grade equivalent of 5.4 and have 
no ability to conduct mathematical operations typically learned 
in* fifth grade* In reality the score indicates a performance as 
goo3 as a fifth grade pupil's would be if that pupil had taken 
thf^ third grade test. 

Normal Curve Eouivalents^ This score is used almost exclusively 
for research purposes. NCEs are normalized standard scores with 
a mean of 50 and a standard deviation of 21.06. They range from 
1*0 ; which is the NCE corresponding to a percentile of 1.0^ to 
99.0^ which is the NCE corresponding to a percentile of 99.0. 
The points between,, however^ are not equivalent. In comparison 
with percentiles^ NvCLs are more spread out toward the middle of 
the distribution and less spread out at the extremes. NCEs are 
the preferaible metric in research ^ but many would argue that they 
do not have truly equal intervals. In essence, NCEs are simply a 
refinement on the stanine score; they -hare the same strengths 
and weaknesses as other normalized standard scores. 

The description of score types illustrates that there are 
difficulties inherent in each of the me< 3ures. While NCEs 
provide a preferable metric to percentiles and grade equivalent 
units, they are not a panacea. They, too, are dependent upon 
ordinal data.* For practical purposes, however, NCEs are treated 
as interval data. 

The theoretical problems of assessing growth or learning 
from test data are, as we have seen, considerable. The 
statistics below are suggested to deal with these difficult 
problems. They are offered as a compromise between what is 
feasible to do in school settings and what can be expected to 
yield valid results. 

Given the lack of consensus in measurement theory, the 
position taken in this guide is that when using tests to infer 
growth or change it is best to compare test results across the 
several available metrics and statistical techniques. 

Using Statistics 

The model comprehensive assessment report presented in 
Appendix 1 includes three common statistics for"" comparing 
aggregate outcome data: a test for the difference between 
proportions, a t-test for independent samples, and a t-test for 
correlated samples. The tests will be described briefly below 
following an introduction to basic statistical concepts. 
Individuals seeking a more complete explanation of the tests, 
^.neir assumptions, and pju'oper applications, or a consideration of 
alternative tests, may wish to consult a standard statistics text 
such as Statistics, 3 ed. by William Lee Hayr (New York, New 
York: CBS College Publishing, 1981) . 
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Basic Statistical Concepts 



Measures of Central Tendency « The mean, median, and mode are 
frequently used to describe the central tendency of a 
distribution. The mode is the most frequently occurring value. 
The medism is the nximber above or below which one h<alf of the 
observations fall. If there are 29 observations, the median is 
the isth largest observation. When there is an even number of 
observations, the median is the mean of the two middle 
observations. The mean ^s the arithmetic average or the sum of 
the value of all observations divided by the number of 
observations. 

Measureri of Dispersion . Two distributions can have the same 
measure of central tendency yet be very different. Consider two 
groups of three individuals in which the average age of the 
participants is 25 years. In the first group, each of the 
members is 25 years old, but in the second group one member is 
50, another is 24 and anot^ier is one' year old. There is more 
dissimilarity in these groups than there is similarity. A 
measure of dissimilarity or dispersion is needed to enhance our 
perception of the groups. 

A commonly used measure is the variance. A variance is 
computed by summing the squared differences from the mean for all 
observations and then dividing by pne less than the number of 
observations. The square root of the variance is called the 
standard deviation. The standard deviation has the appealing 
property of being expressed in the same units of measurement as 
the data itself, rather than in units squared as is the variance. 
The standard deviation of a distribtition is the average absolute 
variation of all test scores around the mean value. For example, 
a standard deviation of 1.40 indicates "on average'* test scores 
varied 1.4 points around the mean. 

Standard Error of Measurement. The standard error of measurement 
indicates how much we would expect a person's score to vary if he 
or she \ 're examined repeatedly with the same test. Since in any 
ordinary situation we have only one score for a person, we 
determine the standard error of measurement based on our 
knowledge of the normal curve and it-s properties. The score is 
derived by multiplying the standard deviation of the test by the 
square root of one minus the reliability coefficient for the 
test. The standard error of measurement is an extremely 
important score for test users. It allows us to sat up 
confidence intervals or a range of score values within which, to 
a certain measurable limit, we can be confident that the true 
score falls. 



61 



84 



statistical Tests for Group Data 



Test scores for individuals are notoriously unreliable, it 
is because of x:hat unrelieUdility that the standard error of 
measurement is such a critical concept for test users. Tests. are 
far more useful when making judgments about groups because group 
data is a more statble measure of performance than scores for 
individual examinees. The following is a description of some of 
the most common statistical tests that can be performed with 
group data* 

D fLfferences between proportions . The niimbers of youngsters 
passing the statewide reference point on standardized tests is 
the primary indicator of achievement in the New York State 
Comprehensive Assessment Report. Users of these data will, 
therefore^ commonly want to know answers to questions such as the 
following: (1) Is tb.ere a difference between the percentage of 
third graders who passed the test reference point in 1986 and the 
numbers of third graders who passed the test reference point in 
1987? (2) Is there a difference between the rate at which a 
certain cohort of youngsters passed the reference point in the 
third grade and sixth grade? 

Solution: We test the difference berween two proportions, 
PI and P2, using the formula stated below. The symbol PI 
represents the percent passing in group one; P2 represents the 
percent passing in group two. The syrbol Ql represents the 
result when PI is subtracted from l.oo; Q2 is 1.00 minus P2. Nl 
refers to the number of subjects in group one and N2 to the 
number in group two. The symbol S is used to represent the 
standard deviation of the test statistic. 



Test Statistic (2) fPl -P2W0 , where 

S(P1-P2) 

ffPlWOl^ fprW02) 
S(P1-P2) =J Nl N2 



For purposes of illustration, let us assxime our interest is 
in determining if there is a difference between the rate of 
passing for two different, but highly comparable, groups of third 
graders; one group entered the system in 1986, the other group 
entered in 1987. We set our variable names as follows: PI is 
the percent of third graders wlio passed the test in 1986 and P2 
is the percent who passed the test in 1987. Again to illustrate 
the point, let us assume .40 passed the reference point in 1986 
and *23 passed the reference point in 1987* Our hypothesis is 
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that the difference between the proportions equals zero (Hq : F3 
• P2 « 0) or alternatively that the difference between the 
proportions does not equal zero (Hi : PI - F2f O) . Given the 
formula and the data for otir problem, the test statistic is 
calculated as fellows: 

(.AO - .23) - 0 



Test Statistic (Z) = / (-40) (.60) + (.27) (.73) 

}l 57 57 

1.93 



For a .05 level of significance and a two-tailed test, the 
decision rule is to reject the null hypothesis (Hq) if Z is 
greater than 1.96 or Z is less than -1.96. Since 1*93 is not 
greater than 1.96, we conclude there is no significant 
difference. 

Two-Sample T-Test for Independent Means. The initial section of 
this appendix sought to demonstrate the theoretical problems 
encoxihtered in attempting to determine differences between test 
scores with correlated samples. While there is some disagreement 
over the use 6f t-tests to ^calculate the difference between 
uncorrelated samples, most statisticians would admit of such a 
use. A typical problem requiring such an application occurs when 
districts are attempting to determine if, for example, the 
difference between the mean achievement of third grade students 
in 1986 and 1987 is significant. The means in this case are 
uncorrelated because the groups are composed of two distinct sets 
of students with no overlapping membership. The solution to this 
problem is given by the formula below. 



Test Statistic (T) 3X1^X2 




The symbol represents the sample mean of a group, S^, the 
variance, and N, the sample ,'iize. 
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Two. Sample T-^Test for Correlated Means , Districts may also be 
interested in knowing if the mean ach ^.evement of the same cohort 
varied between 1986 and 19S7. In this case the means are 
correlated because the same students are in both groups. The 
formula is given below; however^ we caution that the conditions 
for* its application (high reliability of both tests and low 
correlation between the rank ordering of students on the two 
exams) are unlikely to apply, and the resulting information 
should be used cautiously. The symbols used are as follows: D 
is the mean difference between each of the paired scores and 
is the standard deviation of that distribution; the other symbols 
are used as previously described. 

Test Statistic (T for Correlated Samples) « D 



Sd>^*n) where 
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Appendix 3 

Directory of School Improvement Resources 



xlie annotated list of resources below is meant to suppiem^ ^t 
the discUi?sion provided in the text. The selection includes 
ioooks^ manuals^ and organizations that attend (although to 
varyinor degrees) tc the concerns voiced in this guide. The 
primary focus of these resources^ however^ in not with the 
transformation of data into i formation^ but the use of 
information for school improvement purposes, and as such they 
advance cur concerns to their next logical step. For a 
comprehensive annotated listing of school improvement resources, 
we recommend readers consult Implementing School Improvement 
Plans; A Directory of Research-Based Tools (1987) published by 
The Regional Laboratory for Educational Improvement of the 
Northeast and Islands: Andover, Mass. The first selection we 
suggest is taken directly from that source. 



Title ; Improving School Improvement: An Independent 

Evaluation of the California School 
Improvement Program 

Author fs) : Paul Berman and Tom Gjelten 

Description & Purpose This report evaluates California's School 

Improvement Program (SIP) ^ examining t -Aether 
it has been successful and how it mighx: be 
improved. Provides policymakers and citizens 
with basic information and understanding 
about how local instructional improvement can 
and does take place. 

SIP contains many elements of past education 
improvement efforts^ but it also departs from; 
'^hose efforts in these ways: 1) program 
requires schools to adopt elements of a 
"model of change" focused on the change 
process, not on installing a particular 
innovation or instructional approach. 
Diversity in local objectives, plans and 
programs is promoted; 2) involves parents in 
actual program planning^ changing the way 
decisions are made in schools; 3) intends to 
coordinate curriculum and instructional 
programs in a school with the student as 
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fo::us of planning efforts; 4) focuses on 
program quality rather than innovation and 
considers many outcomes in addition to 
student performance; and 5) State review 
provides formative feedback to the schools. 

Consists of 10 chapters that cover these 
areas: assessment of SIP; the effects of 
implementation , school conditions , and 
districts on school improvement; 
implementation process and patterns; 
planning; school site councils and parent 
involvement; staff development; and program 
reviews • 

Cost s free of charge 

Contact Information ! Alex Law, Director 

Program Evaluation and Research Division 
California State Department of Education 
P. 0. Box 944272 
Sacramento , CA 94244-2720 
(916) 322-5010 
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Title; 



Bulldlr4g Effective Schools: Assessing, 
PI ann Ing , Imp 1 ement Ing 



Author fs) t Janet Chrlspeels and David Meaney, Ed.D. 

Description & Purpose : Manual is designed to assist school 

administrators, principals, and other 
educational leaders who wish to implement a 
school improvement program based on the 
effective schools research. 

Defines the term "effective school," 
describes characteristics associated with 
effective schools, and outlines the process 
being used by the San Diego County Office of 
Education to assist schools in becoming 
effective. The process is ba^ed on research 
on effective schools, effective classroom 
teaching, organizational development, and 
educational change • 

Goal of the program is to increase the 
overall level of student achievement for all 
students through a process of assessment and 
planning and the implementation of a site- 
generated school effectiveness plan. 
Purposes are: 1) to determine the existence 
of specified elements needs to achieve school 
effectiveness; 2) to assess the school's 
academic effectiveness; 3) to determine the 
school's climate; 4) to establish a school 
action plan for improvement; and 5) to 
implement a plan for :.mprovement . 

Consists of four chapters and tables, charts'^ 
and appendices that include the following 
materials that could be useful staff 
development handouts : teacher/staff 
questionnaires, parent/student surveys- 
assessment instrximents , sample comparisons, 
interview questions, evaluation forms, 
numerous test results, data collection forms, 
observation forms, worksheets and techniques. 

Cost : ' $20.00 

Contact Information : . San Diego County Office of Education 

Room 212 

6401 Linda Vista Road 
San Diego, CA 94211-7399 
(619) 292-3500 
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All Our Kids Can Learn to Read: A Guide to 
Parent and Citizen Action 



Development by Chicago SCHOOLWATCH^ a product 
of Designs for Change^ Chicago^ Illinois 

Intended to be used as a handbook by parents, 
educators^ citizens^ and otuers who wish to 
take an active part in school improvement. 
Although there is an emphasis on improving 
reading skills^ All Our Kids Can Learn to 
Read is very helpful in planning and 
implementing change in schools and can be 
applied to various sxibject areas and school 
improvement projects and issues. 

Consists of five chapters that take an 
innovative approach to developing ingredients 
needs to teach/1 eam. Appendices include 
reading selections and principal interviews. 
Suggested program evaluation checklists and 
examples are included. 

Chapter 1 is an introduction to SCHOOLWATCH 
and the Campaign for Effective Schools in 
Chicago, Illinois; Chapter 2, Are your Kids 
Learning to Read? analyzes skills needed to 
master the skill of reading (includes 
testing, evaluation, and the Chicago Mastery 
Learning Reading Program); Chapter 3, How 
Does YOur School Measure Up? describes ten 
ingredients identified as necessary for an 
effective school and includes the 
"SurtuOLWATCH Report Card" (checklist) that 
specifies what to look for when evaluating 
progress in school improvement; Chapter 4, 
What About the Bureaucracy? stresses 
decentralizing the bureaucracy and changes 
that can be made to improve how it works for 
the school; Chapter 5, Action, suggests 
active approaches to effective school 
improvement efforts. 

Also available in Spanish. 

$3.50 

Designs for Change 
220 South State Street 
Chicago, IL 60604 
(312) 922-0317 
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Instructional Leadership Handbook 



James W. Keefe and John M. Jenkins, Editors 

Intended to be used as a reference book by 
principals working to become more effective 
instructional leaders. 

The handbook is organized according to the 
role of the principal as instructional 
leader, that of providing direction, 
resources, and support to teachers and 
students for the improvement of teaching and 
learning in the school. The handbook reviews 
that in four areas: formative; planing; 
implementation; and evaluation. 

Serves as a comprehensive resource for the 
principal who is learning to be a more 
effective instructional leader, and as a tool 
for use in staff inservice activities. 

Formative Elements discusses trends in: 
content fields, organization and staffing, 
and media and methods; Planning Elements 
discusses 10 elements of effective planning, 
from needs assessment to budgeting for 
instructional improvement; Implementation 
Elements discusses one organizational and 
four supervisory elements: organizing the 
program, supervising classroom management, 
supervising the diagnostic process, 
supervii Ing prescription and placement, and 
supervising instruction; Evaluation Elements 
lists six evaluatory elements: assessing and. 
reporting student progress, teacher 
performance appraisal, program evaluation, 
community feedback, program modification and 
revision, and communicating the program* 

$8.00 

Pxiblications Department 

National Assn. Secondary School Principals 
1904 Association D7:ive 
Reston, VA 22091 
(703) 860-0200 
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Title: A Process Guide for school Improvement 

Author (s) : Herbert J. Klausmeier 

Description & Purpose : Guide is for use in locally conducted 

inservice programs and in university courses 
and programs. It is designed to aid 
individual schools as well as district 
ofiices in starting and maintaining a self- 
improvement capability that is characterized 
by: 1) maintaining satisfactory student 
outcomes and improving unsatisfactory ones; 
2) maintaining or improving effective 
instructional practices; 3) maintaining or 
improving job satisfaction and morale; and 4) 
increasing competency and professional 
development of staff. Chapter 1 is for all 
levels of schools, Chapters 2 through 11 are 
more for the middle school and high school, 
and Chapter 12 is more the elementary school. 
Chapters 2 through : ^ give improvement 
suggestions and illustrative exemplary 
practices of both middle schools and high 
schools. Chapter 12 does the same for 
elementary schools. Most information 
regarding middle schools or elementary 
schools is relevant to both. 

Each chapter includes an improvement plan, 
covering such areas as: 
administration/structural organization; 
program planning; curriculxim planning; school 
climate; testing/data gathering/ evaluation; 
teacher-advisor program planning; 
home/school/community relations; and 
components contributing to effectiveness and 
activity planning to increase effectiveness. 
In addition, the Appendix offers suggestions 
for using the guide in locally conducted 
inservice/staff development programs and 
s plies supplementary materials for 
secondary and elementary schooling. Also, 
simulations are available from the Wisconsin 
Center for Education Research (one each — 
elementary, middle and high sc;hool) . 

Cost ; $12.75 paperback, $28.00 hardcover 

Contact Information : University Press of America 

4720 Boston Way 
Lanham, MD 20706 
(301) 459-3366 
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Title ; 
Author fs) : 

Description & Purpose : 



Materials : 
Cost: 

Contact Information; 



Reaching for Excellence: An Effecti 'e 
Schools Sourcebook 

Regina M. J. Kyle, Ed. 

An integrated document that contains 
summaries of t:;ie knowledge base as well as a 
current directory of programs and sources of 
technical assistance relevant to research and 
practical information in the area of school 
effectiveness/improvement programs being 
implemented in the United States. * Intended 
for use by administrators, curriciilum 
specialists, staff developers, and tee *iers. 

The overview of the book emphasizes key 
themes and issues across the chapters on 
research that appear in Part I, including an 
extended discussion of staff development, and 
addresses the process of translating research 
into policy with illustrations. 

Six of the seven chapters summarize the 
research from these perspectives: effective 
classroom practices in elementary schools and 
secondary schools; effective school practices 
at each level; district- and state-level 
practices that support effective school 
management and instruction. Includes a 
chapter on criteria and methods for measuring 
effectiveness. Part II presents a directory 
of successful effective schools programs 
being currently implemented. 

Directory of Programs Promoting Effective 
Practices at the Classroom and Building 
Levels 

• • 

$9.50 

Superintendent of Documents 
U. S. Government Printing Office 
Washington, D.C. 20402 
(202) 783-3238 
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Title ; 

Author (s) : 

Description & Purpose : 



Description ; 



Cost ; 

Contact Information; 



The Practice Profile; An All-Purpose Tool 
for Program Communication^ Staff Development^ 
Evaluation^ and Improvement 

Susan Loacks-Horsley and David P. Crandall 

Provides a standardized, systematic, and 
cost-effective way to summarize program 
components and requirements. Provides 
guidance as to implementing and evaluating 
the practice and facilitating comparison with 
other programs. Especially helpful to 
program/project directors, staff developers, 
decis ionmaker s , evaluators , potential 
adopters, and researchers. 

Describes the three parts of a Practice 
Profile; a component checklist contains 
components that describe the practice in use; 
a listing of implementation requirements; and 
an assessment of practice characteristics. 

Serves as a systematic means of defining 
selected practices. Explains the need for 
and the parts of such a practice profile, and 
suggests a procedure for the collection of 
data and development of an accurate profile. 

$3.00 

Cynthia Connolly 
The NETWORK, Inc. 
290 South Main Street 
Andover, MA 01810 
(617) 470-1080 
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Title ; An Action Guide to School Improvement 

Author fs) ; Susan Loucks-Horsely and Leslie F. Hergert 

Description S Purpose : Intended primarily for educators who are 

responsible for local school improvement 
(principals, coordinators, teachers, 
department heads, administrators) ; also for 
trainers, outside support (such as 
consultants and agencies) , and decision 
makers (e.g., local school boards, education 
agencies, and legislators). 

Describes a practical, research-based 
conceptual framework for implementing change 
to bring about school improvement. 

The guide is divided into seven linear steps 

1. Establishing the School Improvement 
Pro j ect 

2. Assessment and Goal Setting 

3. Identifying an Ideal Solution 

4. Preparing for Implementation 

5 . Implementing 

6. Review 

7. Maintenance and Institutionalization 

An appendix is provided, with sources of 
school improvement resources that are 
available to schools. 

Cost : $5.00 

Contact InforTnation ; Cynthia Connolly 

The Regional Laboratory for Educational 
Improvement of the Northeast and Islands 
290 South Main Street 
Andover, MA 01310 
(617) 470-0098 
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Title ; 



Achieving Excellence (A+) 



Sponsor ; 



Mid-Continent Regional Educational Laboratory 



Description S Purpose : 



The Achieving Excellence (A+) Program 
provides a management tool that helps 
educators take stock of their current 
practices in light of current research. A+ 
suggests performance indicators that local 
personnel use to, gather baseline data. These 
indicators are then used to monitor the 
impact of improvement strategies. 



Materials ; 



Three, ring binder Achieving Excellence^ 
divided into three main sections: Academic 
Efficiency^ Student Success^ Improvement 
Management . 



Cost; 



Dependent upon number of participants and 
design of program^ on a district-by district 
basis; binder can be purchased separately for 
$50.00. 



Contact Information; 



Toni Hass 

12500 East Iliff Avenue 
Suite 201 
Aurora^ CO 80014 
(303) 337-0990 
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Titles; 



How to Design a Program Evaluation 
How to Measure Achievement 
How to Measure Program Implementation 
How to Present An Evaluation Report 



Author fs) ; 



Lynn Lyons Morris & Carol Taylor Fitz-Gibbon 



Description & Purpose : 



Guides and assists practitioners' (at all 
levels in planning and managing of 
experience) evaluations . 

Consists of four selections from the Program 
Evaluation Kit (Sage Publications^ Beverly 
Hills) . The complete Program Evaluation kit 
contains eight books that answer questions 
that a practicing evaluator might ask. 

These four practical guides have been field 
tested and offer details advice^ clear 
definitions^ and useful procedures^ in non- 
technical language. 



Cost: 



Can be purchased as a set ($59.95) or 
individually: 

How to Design a Program Evaluation - $8.50 
How to Measure Achievement - $8.50 
How to Measure Program Implementation - $7.9 
How to Present An Evaluation Report - $4*95 



Contact Information ; 



Sage Publications 
275 South Beverly Drive 
Beverly Hills, CA 90212 
(212) 274-8003 
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Title : 



Research Summaries and a Director of 
Programs, Services, and Resources 



Author fs) ; Lawrence W. Lezotte and Stuart C. Rankin 

Description & Purpose : This document is the outcome of the project 

titled, "Research Dissemination Through 
* CollaQDorat5.ve Planning for School 
Improvement" (NIE) . The first section of the 
book contains 36 summaries of research 
helpful to anyone involved in the school 
improvement planning process^ for use on any 
level by educators within the school system, 
and by consultants, trainers, or outside 
evaluators . Sections include: Adminis- 
trative-Instructional Lejadership; Monitoring 
Pupil Progress; School Climate; School Goals 
and Resources/Basic School Skills; Teacher 
Expectations; Classroom Management; 
Instructional Strategies; and Home-School 
Relations. 

The second section contains six case studies 
that track the project as it occurred in each 
of the six schools (three elementary level, 
two middle schools and one high school) . 
Provides descriptions, and contains lessons 
helpful at any stage of the school 
improvement process, from planning to 
implementation to evaluation. 

The final section contains resources (local) 
that illustrate how individual schools might 
develop a directory of available program 
resources • 

Materials: Case studies, evaluation checklists, planning 

outlines. Directory of Programs. Services, 
and Resources . School Improvement 
Bibliography , Publications List (The 
Institute for Research on Teaching) • 

Cost: $15.00 (check made payable to Michigan State 

University) 

Contact Information ; Lawrence W. Lezotteor Stuart C. Rankin 

Effective Schools Office 
403 Erickson 
College of Education 
Michigan State University 
East Lansing, MI 
(571) 353-6413 
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2itle: Developing and Institutionalizing a Self- 

Improvement Capability: Structures and 
Strategies of Secondary Schools 

Author fs) : Herbert J. Klausmeier 

Description & Purpose : Developing and Institutionalizing a Self- 
Improvement Capability (September 1982 - 
November 1984 study) primarily clarifies the 
institutionalization of a school's self- 
improvement capetbility and also validates the 
improvement strategies, faciiitative organi- 
zational structures, and support arrangements, 
involved in a school's initial development of 
a self- improvement capability. Includes 10 
schools (elementary, middle, high school) who 
annually collected student performance data 
for 1981-82 and then 1533-84 (academic 
achievement in one or more ^areas' as measured 
by standardized tests, average daily 
attendance, incidence of discipline referrals 
and suspensions). Reporting forms, 
observations, checklists^, and interviews were 
used to gather data. Forms are available 
from the Wisconsin Center for Educational 
Research. Eight organizational structures and 
support arrangements were validated as 
facilitating the implementation of the three 
improvement strategies: 1) organization of 
principal and representative teachers into an 
effectively fvihctioning leadership- 
coordinating group. This is most critical 
because it is essential for implementing the 
goal setting strategy and is prerequisite to 
effective functioning of the remaining seven; 
2) organization of teachers and students into 
small groups; 3) teachers advising students; 
4) scheduling of classes so that mutual 
interests have a common time during the 
school day for planning improvement 
activities; 5) principal's effective 
instructional leadership; 6) ongoing 
inservice/staff development; 7) support of 
district officials; 8) school's autonomy to 
implement improvement program; and 9) parent 
participation and support. 

Cost : , $12.25 

Contact Information : University Press of America 

4720 Boston Way, Lanham, MD 20706 
(301) 359- 66 
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Additional Books 



Education USA: Special report^ Good Schools; What Makes Them Work 
(Virginia: National School Public Relations Association) , 1981. 

Brief bock that provides clear and concise overview of the 
literature^ philosophy^ and procedures of effective schools 
movement • 

Edgar A. Kelly^ Improving School Climate: Leadership Techniques for 
Educators. (National Association of Secondary School Principals)^ 1980. 

The central theme of this monograph is that school can make a 
difference in what happens to the people who work and study in 
them. Focusing on the role of the principal in. affecting school 
climate, it deals most directly with assessing school climate and 
planing for its improvement. 

Rolf Lehming and Michael Kane (ed.). Improving Schools: Using What We 
Know (New York: Sage Publications) , 1981. 

Series of articles written by such noted people as Ernest House and 
Matthew Miles, the work was developed under a contract with the Far 
West Laboratory and funding was provided by NIE. The articles are 
concerned with innovation and organizational change in the schools. 

George F. Madau, Peter W. Airasian, and Thomas Kellaghan, School 
Effectiveness; A Reassessment of the Evidence (New York: McGraw-Hill) , 
1980. 

This book discusses some of the major issues in determining what 
about schools really makes a difference in student learning. Some 
of the key literature in local school improvement is summarized. 

Carl L. Marburger^ One School at a Time (Columbia^ MD: 
The National Committee for Citizens in Education)^ 1985. 

Written by one of the key figures in the movement^ this book is 
designed to familiarize various m(ambers of the school community 
with the ideas and concepts of school-based management and help 
them to get started with its implementation. 

Daniel C. Neale^ William J. Bailey^ and Billy E. Ross^ Strategies for 
School Improvement: Cooperative Planning and Organization Development 
(Boston: Allyn and Bacon) , 1981. 

This book "written especially for educators who are seeking a 
careful sximmary of current knowledge about planned change in 
ecTucation/' focuses on the local school-building organization and 
collaboration among all members of the local school community as 
key ingredients for change. Using the "Partnership Model for 
School Improvement^" topics such as the tactics and methods of 
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school improvement are discussed. Bibliographies, planning guides, 
and questions for discussion are included. 



Manuals 



California State Department of Education, Assistance Guide for Forming a 
Consortium to Improve School Programs (Sacramento, CA: State Department 
of Education) , 1981. 

Manual for districts covers all facets of consortium formation* 

National Study of School Evaluation, Evaluation Criteria: Fifth Edition 
(Virginia: NSSE) , 1978. 

The manual is based *'on the principle that a school should be 
judged in terms of what it is striving to achieve and according to 
the extent to which it is meeting the needs of the students 
enrolled and the community it serves." It provided instructions, 
procedures, and sample forms for self-*evaluation. 

Seattle Public Schools, School-Based Planning Manual . 1982. 

A step-by-step planning guide including forms and checklists. 



organizational Resources - Regional Educational Laboratories, R&D 
Cenvters. Universities 



The Academy for Educational Development, School Services Division, 680 
Fifth Avenue, New York, NY 10019 Telephone (212) 397-0040. 

Technical assistance and resource materials are available. 

Iowa state University, School Improvement Model, College of Education, 
Ames, lA 50011, Telephone: (515) 294-5521 

A method of school-based assessment developed from the effective 
schools literature. Consulting services may be contracted. 
Extensive manuals and related litera\:ure are available. 

National Study of School Evaluation, 5201 Leesburg Pike, Falls Church, 
VA 22041. 

Can provide such resources as evaluation manuals, and inventories 
of parent, teacher, and student opinions. 
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Northwest Regional Educational Laboratory, 300 SW Sixth Ave., Portland, 
OR 97204, Telephone; (800) 547-6339. 

Onward to Excellence Program was developed to train leadership 
teams in schools to apply effective schools research and goal based 
management practices to local improvement efforts. Program 
descriptions, manuals, and resource lists are available. 

Research for Better Schools (RBS) , 444 North Third St., Philadelphia, PA 
19123, Telephone: (215) 574-9300. 

The Secondary School Development Program and School Effectiveness 
Training Program aim to improve effectiveness at the school 
building level. RBS provides technical assistance, materials, 
manuals, and survey instruments. These programs are jointly 
sponsored by the New Jersey Education Association and Pennsylvania 
State Education Association. 

The Regional Laboratory for Educational Improvement of the Northeast and 
Islands, 290 South Main Street, Andover, MA 01810, Telephone: (617) 470- 
0098. 

Technical assistance and resource materials are available. 



Orarmization Resources - School Districts and State Departments of 
Education 



California State Department of Education, Office of School Improvement, 
721 Capitol Mall, Sacramento, CA 95814, Telephone: (916) 322-5954. 

One of the oldest and most extensive statewide efforts. Available 
materials include handbooks, manuals, and evaluation reports. 

Chicago Public Schools, Chicago Effective Schools Project, 1819 W. 
Pershing Road, East Center 6, Chicago, IL 60609, Telephone: (312) 890- 
7771. 

In operation i;ince 1981, the program strives to apply effective 
schools research in instituting a program to lessen the educational 
disadvantage that accrues to minority students in racially isolated 
schools. They can provide technical assistance, plans, and 
evaluation reports. 

New York City Public Schools, Office of Comprehensive School Improvement 
Projects (OCSIP) , 110 Livingston Street, 'Brooklyn, NY, Telephone: (718) 
935-4055. 

Based on the principles of school effectiveness as articulated by 
such researchers as Ron Edmonds, the program has been implemented 
in elementairy and middle schools throughout the city. Technical 
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assistance and instruments are available. In addition evaluation 
reports and documentation instruments are available through the 
Office of Educational Assessment, 110 Livingston Street, 7th Floor 
Brooklyn, IC£ 11201. 
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